[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:15 dist-test-slave-vkjp krb5kdc[23912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:15 dist-test-slave-vkjp krb5kdc[23912](info): set up 2 sockets
Aug 05 23:15:15 dist-test-slave-vkjp krb5kdc[23912](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260805 23:15:16.945861 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:17.083899 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43833
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:43833
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:17.209937 23925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:17.210279 23925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:17.210369 23925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:17.214507 23925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:17.214617 23925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:17.214655 23925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:17.214695 23925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:17.214733 23925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:17.220439 23925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:43833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:17.222038 23925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:17.223412 23925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:17.237860 23933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:17.238067 23930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:17.238376 23931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:17.238626 23925 server_base.cc:1034] running on GCE node
I20260805 23:15:17.239527 23925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:17.241412 23925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:17.242620 23925 hybrid_clock.cc:648] HybridClock initialized: now 1785971717242558 us; error 78 us; skew 500 ppm
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:17.245903 23925 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:17.247208 23925 webserver.cc:460] Webserver started at http://127.23.88.126:38477/ using document root <none> and password file <none>
I20260805 23:15:17.248199 23925 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:17.248306 23925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:17.248742 23925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:17.251757 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "b83b339559b4435f9445026b0c25f95c"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "1e0567aa96e4e77e1c2db4cf339b3c4d"
server_key_iv: "f9dc340eb2a690c8e1834e6a337737f0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.252514 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "b83b339559b4435f9445026b0c25f95c"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "1e0567aa96e4e77e1c2db4cf339b3c4d"
server_key_iv: "f9dc340eb2a690c8e1834e6a337737f0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.258067 23925 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260805 23:15:17.262007 23940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.263835 23925 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260805 23:15:17.264073 23925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "b83b339559b4435f9445026b0c25f95c"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "1e0567aa96e4e77e1c2db4cf339b3c4d"
server_key_iv: "f9dc340eb2a690c8e1834e6a337737f0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.264230 23925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:17.291142 23925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:17.295243 23925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:17.295539 23925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:17.310870 23925 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:43833
I20260805 23:15:17.310887 23994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:43833 every 8 connection(s)
I20260805 23:15:17.312515 23925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:17.318017 23995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:17.322165 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 23925
I20260805 23:15:17.322376 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:17.323927 23905 external_mini_cluster.cc:1428] Setting key 342f4d80bccecd5436079ee519b11667
I20260805 23:15:17.328619 23995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c: Bootstrap starting.
I20260805 23:15:17.335110 23995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:17.337517 23995 log.cc:824] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:17.341500 23995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c: No bootstrap required, opened a new log
I20260805 23:15:17.349474 23995 raft_consensus.cc:348] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } }
I20260805 23:15:17.349943 23995 raft_consensus.cc:374] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:17.350064 23995 raft_consensus.cc:729] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83b339559b4435f9445026b0c25f95c, State: Initialized, Role: FOLLOWER
I20260805 23:15:17.351023 23995 consensus_queue.cc:260] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } }
I20260805 23:15:17.351313 23995 raft_consensus.cc:388] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:17.351459 23995 raft_consensus.cc:482] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:17.351626 23995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 0 FOLLOWER]: Advancing to term 1
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:17.353314 23995 raft_consensus.cc:504] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } }
I20260805 23:15:17.353925 23995 leader_election.cc:302] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b83b339559b4435f9445026b0c25f95c; no voters:
I20260805 23:15:17.354334 23995 leader_election.cc:288] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:17.354521 24000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:17.354775 24000 raft_consensus.cc:686] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [term 1 LEADER]: Becoming Leader. State: Replica: b83b339559b4435f9445026b0c25f95c, State: Running, Role: LEADER
I20260805 23:15:17.355299 24000 consensus_queue.cc:237] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } }
I20260805 23:15:17.356442 23995 sys_catalog.cc:564] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:17.358738 24002 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b83b339559b4435f9445026b0c25f95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } } }
I20260805 23:15:17.358897 24002 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:17.359311 24001 sys_catalog.cc:455] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b83b339559b4435f9445026b0c25f95c. Latest consensus state: current_term: 1 leader_uuid: "b83b339559b4435f9445026b0c25f95c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b83b339559b4435f9445026b0c25f95c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43833 } } }
I20260805 23:15:17.359673 24001 sys_catalog.cc:458] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:17.359921 24009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:17.363204 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:17.330739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37148 (local address 127.23.88.126:43833)
0805 23:15:17.331286 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:17.331328 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:17.331387 (+ 59us) server_negotiation.cc:407] Connection header received
0805 23:15:17.338444 (+ 7057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:17.338491 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:17.339126 (+ 635us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:17.339455 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:17.343791 (+ 4336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.345178 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.347355 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.348121 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.348265 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:17.354224 (+ 5959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:17.354266 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:17.354279 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:17.354343 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:17.357505 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:17.358444 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:17.358453 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:17.358461 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:17.358678 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:17.359248 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:17.359252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:17.359254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:17.360046 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:17.360313 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:17.361422 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0805 23:15:17.361808 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":164,"threads_started":1}
I20260805 23:15:17.365895 24009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:17.375892 24009 catalog_manager.cc:1347] Generated new cluster ID: 3302775f25b842f697e89fa4e9ba9c33
I20260805 23:15:17.375994 24009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:17.389371 24009 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:17.390558 24009 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:17.396348 24009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b83b339559b4435f9445026b0c25f95c: Generated new TSK 0
I20260805 23:15:17.397269 24009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:17.434928 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:17.592955 24021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:17.593461 24021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:17.593540 24021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:17.598330 24021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:17.598438 24021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:17.598549 24021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:17.604792 24021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:17.606061 24021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:17.607204 24021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:17.619961 24028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:17.619941 24031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:17.619942 24026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:17.620934 24021 server_base.cc:1034] running on GCE node
I20260805 23:15:17.621470 24021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:17.622140 24021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:17.623312 24021 hybrid_clock.cc:648] HybridClock initialized: now 1785971717623295 us; error 37 us; skew 500 ppm
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:17.627377 24021 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:17.628414 24021 webserver.cc:460] Webserver started at http://127.23.88.65:36385/ using document root <none> and password file <none>
I20260805 23:15:17.629098 24021 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:17.629171 24021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:17.629406 24021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:17.631883 24021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "2983d03f3bc740d6a8e78f28787f4f57"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "d6d51b9166090ac264af4b355fad5ad6"
server_key_iv: "41e5bfc7aff34d45e0198eb582afbee0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.632685 24021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "2983d03f3bc740d6a8e78f28787f4f57"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "d6d51b9166090ac264af4b355fad5ad6"
server_key_iv: "41e5bfc7aff34d45e0198eb582afbee0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.637778 24021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260805 23:15:17.641103 24039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.642941 24021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:17.643180 24021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "2983d03f3bc740d6a8e78f28787f4f57"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "d6d51b9166090ac264af4b355fad5ad6"
server_key_iv: "41e5bfc7aff34d45e0198eb582afbee0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.643347 24021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:17.667985 24021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:17.671821 24021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:17.672111 24021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:17.673062 24021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:17.674423 24021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:17.674480 24021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.674556 24021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:17.674587 24021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.698675 24021 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:33373
I20260805 23:15:17.698932 24160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:33373 every 8 connection(s)
I20260805 23:15:17.699820 24021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:17.704623 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24021
I20260805 23:15:17.704859 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:17.705156 23905 external_mini_cluster.cc:1428] Setting key fcff31bb4c2320e84e85611f758770fc
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:17.718760 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:17.702467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:56847 (local address 127.23.88.126:43833)
0805 23:15:17.702646 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:17.702651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:17.704181 (+ 1530us) server_negotiation.cc:407] Connection header received
0805 23:15:17.705505 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:17.705510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:17.705574 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:17.705680 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:17.707607 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.708595 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.710205 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.710632 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.710776 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:17.714724 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:17.714727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:17.714729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:17.714759 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:17.716401 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:17.717013 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:17.717022 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:17.717023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:17.717110 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:17.717509 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:17.717513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:17.717516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:17.717727 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:17.717841 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:17.718429 (+ 588us) server_negotiation.cc:299] Negotiation successful
0805 23:15:17.718574 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:15:17.720075 24163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:17.702828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43833 (local address 127.23.88.65:56847)
0805 23:15:17.703905 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:17.703953 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:17.705208 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:17.705946 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:17.705953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:17.706527 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:17.707430 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:17.707449 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.708784 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.708787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:17.710053 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:17.710064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:17.710852 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:17.710855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:17.711174 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:17.712027 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:17.712053 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:17.714486 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:17.716554 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:17.716562 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:17.716581 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:17.716884 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:17.717213 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:17.717217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:17.717219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:17.717388 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:17.717843 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:17.717853 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:17.718161 (+ 308us) client_negotiation.cc:769] Sending connection context
0805 23:15:17.718366 (+ 205us) client_negotiation.cc:240] Negotiation successful
0805 23:15:17.718802 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"thread_start_us":239,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:17.722194 24161 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43833
I20260805 23:15:17.722553 24161 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:17.723248 24161 heartbeater.cc:511] Master 127.23.88.126:43833 requested a full tablet report, sending...
I20260805 23:15:17.726397 23958 ts_manager.cc:194] Registered new tserver with Master: 2983d03f3bc740d6a8e78f28787f4f57 (127.23.88.65:33373)
I20260805 23:15:17.728164 23958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:56847
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:17.743894 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:17.906423 24167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:17.906731 24167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:17.906807 24167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:17.911190 24167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:17.911271 24167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:17.911372 24167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:17.917061 24167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:17.918208 24167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:17.919261 24167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:17.929412 24173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:17.929943 24167 server_base.cc:1034] running on GCE node
W20260805 23:15:17.931412 24172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:17.931828 24176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:17.932392 24167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:17.933082 24167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:17.934262 24167 hybrid_clock.cc:648] HybridClock initialized: now 1785971717934244 us; error 30 us; skew 500 ppm
Aug 05 23:15:17 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:17.937364 24167 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:17.938556 24167 webserver.cc:460] Webserver started at http://127.23.88.66:37593/ using document root <none> and password file <none>
I20260805 23:15:17.939193 24167 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:17.939265 24167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:17.939477 24167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:17.941664 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "99cc3ec16e694817a536134829e7c565"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "822a6c4c4093fb6f8fece33a02b2d054"
server_key_iv: "c2e1694ae0d5eab5ddcaa5884cf50fa0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.942341 24167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "99cc3ec16e694817a536134829e7c565"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "822a6c4c4093fb6f8fece33a02b2d054"
server_key_iv: "c2e1694ae0d5eab5ddcaa5884cf50fa0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.946583 24167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260805 23:15:17.950073 24183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.951851 24167 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:17.952008 24167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "99cc3ec16e694817a536134829e7c565"
format_stamp: "Formatted at 2026-08-05 23:15:17 on dist-test-slave-vkjp"
server_key: "822a6c4c4093fb6f8fece33a02b2d054"
server_key_iv: "c2e1694ae0d5eab5ddcaa5884cf50fa0"
server_key_version: "encryptionkey@0"
I20260805 23:15:17.952163 24167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:17.967675 24167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:17.971069 24167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:17.971406 24167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:17.972123 24167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:17.973090 24167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:17.973140 24167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.973181 24167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:17.973235 24167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:17.992588 24167 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:37443
I20260805 23:15:17.992800 24300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:37443 every 8 connection(s)
I20260805 23:15:17.993995 24167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:18.003372 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24167
I20260805 23:15:18.003480 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:18.003746 23905 external_mini_cluster.cc:1428] Setting key a80046666ab9d145a5c6c9102898fa7e
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971717, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:18.011765 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:17.996247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46995 (local address 127.23.88.126:43833)
0805 23:15:17.996401 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:17.996405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:17.997862 (+ 1457us) server_negotiation.cc:407] Connection header received
0805 23:15:17.999008 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:17.999010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:17.999072 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:17.999169 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:18.000768 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.001339 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.002746 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.003004 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.003177 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.006899 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:18.006902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:18.006904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:18.006932 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:18.009126 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.009774 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.009778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.009781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.009888 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.010294 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.010300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.010305 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.010623 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:18.010736 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:18.011333 (+ 597us) server_negotiation.cc:299] Negotiation successful
0805 23:15:18.011502 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:18.012601 24303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:17.996648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43833 (local address 127.23.88.66:46995)
0805 23:15:17.997694 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:17.997742 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:17.998777 (+ 1035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:17.999397 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:17.999405 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:17.999957 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:18.000608 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:18.000625 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.001485 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.001488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:18.002606 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:18.002617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.003271 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.003275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:18.003580 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.004460 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:18.004477 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:18.006634 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:18.009290 (+ 2656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:18.009296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:18.009310 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:18.009648 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:18.009992 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:18.009994 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:18.009995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:18.010151 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:18.010735 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:18.010741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:18.011012 (+ 271us) client_negotiation.cc:769] Sending connection context
0805 23:15:18.011248 (+ 236us) client_negotiation.cc:240] Negotiation successful
0805 23:15:18.011551 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"thread_start_us":212,"threads_started":1}
I20260805 23:15:18.013943 24301 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43833
I20260805 23:15:18.014284 24301 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:18.015166 24301 heartbeater.cc:511] Master 127.23.88.126:43833 requested a full tablet report, sending...
I20260805 23:15:18.016619 23958 ts_manager.cc:194] Registered new tserver with Master: 99cc3ec16e694817a536134829e7c565 (127.23.88.66:37443)
I20260805 23:15:18.017233 23958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:46995
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:18.039669 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:18.195288 24306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:18.195636 24306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:18.195721 24306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:18.200367 24306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:18.200503 24306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:18.200605 24306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:18.206926 24306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43833
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:18.208143 24306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:18.209231 24306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:18.221011 24312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:18.221095 24314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:18.221016 24311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:18.221248 24306 server_base.cc:1034] running on GCE node
I20260805 23:15:18.221854 24306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:18.222496 24306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:18.223682 24306 hybrid_clock.cc:648] HybridClock initialized: now 1785971718223667 us; error 27 us; skew 500 ppm
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:18.227337 24306 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:18.228322 24306 webserver.cc:460] Webserver started at http://127.23.88.67:36215/ using document root <none> and password file <none>
I20260805 23:15:18.228863 24306 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:18.228920 24306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:18.229094 24306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:18.231388 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "985b97eacb6c4a078c1252d087bd354f"
format_stamp: "Formatted at 2026-08-05 23:15:18 on dist-test-slave-vkjp"
server_key: "c79a647bcef325c4414871976a9d18aa"
server_key_iv: "73b7018b9221b07145bd0df12f91ef72"
server_key_version: "encryptionkey@0"
I20260805 23:15:18.232036 24306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "985b97eacb6c4a078c1252d087bd354f"
format_stamp: "Formatted at 2026-08-05 23:15:18 on dist-test-slave-vkjp"
server_key: "c79a647bcef325c4414871976a9d18aa"
server_key_iv: "73b7018b9221b07145bd0df12f91ef72"
server_key_version: "encryptionkey@0"
I20260805 23:15:18.236338 24306 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:15:18.238979 24321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:18.240332 24306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:15:18.240478 24306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "985b97eacb6c4a078c1252d087bd354f"
format_stamp: "Formatted at 2026-08-05 23:15:18 on dist-test-slave-vkjp"
server_key: "c79a647bcef325c4414871976a9d18aa"
server_key_iv: "73b7018b9221b07145bd0df12f91ef72"
server_key_version: "encryptionkey@0"
I20260805 23:15:18.240597 24306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:18.274845 24306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:18.278913 24306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:18.279227 24306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:18.280076 24306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:18.281075 24306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:18.281124 24306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:18.281194 24306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:18.281231 24306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:18.299574 24306 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:38085
I20260805 23:15:18.299597 24441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:38085 every 8 connection(s)
I20260805 23:15:18.300594 24306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:18.308910 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24306
I20260805 23:15:18.309058 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:18.309338 23905 external_mini_cluster.cc:1428] Setting key edb04e51e4d90fee6b625bbd40b73280
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:18.317304 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:18.302955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:51227 (local address 127.23.88.126:43833)
0805 23:15:18.303156 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:18.303161 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:18.304492 (+ 1331us) server_negotiation.cc:407] Connection header received
0805 23:15:18.305710 (+ 1218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:18.305713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:18.305763 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:18.305843 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:18.307390 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.307927 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.309758 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.309976 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.310099 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.313552 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:18.313556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:18.313559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:18.313594 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:18.315157 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.315702 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.315706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.315710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.315794 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.316156 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.316159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.316161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.316360 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:18.316472 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:18.316998 (+ 526us) server_negotiation.cc:299] Negotiation successful
0805 23:15:18.317129 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:15:18.318217 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:18.303425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43833 (local address 127.23.88.67:51227)
0805 23:15:18.304298 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:18.304374 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:18.305507 (+ 1133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:18.306024 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:18.306031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:18.306585 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:18.307240 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:18.307255 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.308075 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.308078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:18.309622 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:18.309635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.310369 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.310373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:18.310722 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.311588 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:18.311606 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:18.313373 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:18.315280 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:18.315286 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:18.315303 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:18.315588 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:18.315890 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:18.315893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:18.315895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:18.316050 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:18.316477 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:18.316485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:18.316743 (+ 258us) client_negotiation.cc:769] Sending connection context
0805 23:15:18.316930 (+ 187us) client_negotiation.cc:240] Negotiation successful
0805 23:15:18.317271 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":184,"threads_started":1}
I20260805 23:15:18.319630 24442 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43833
I20260805 23:15:18.319890 24442 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:18.320406 24442 heartbeater.cc:511] Master 127.23.88.126:43833 requested a full tablet report, sending...
I20260805 23:15:18.321496 23958 ts_manager.cc:194] Registered new tserver with Master: 985b97eacb6c4a078c1252d087bd354f (127.23.88.67:38085)
I20260805 23:15:18.322234 23958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:51227
I20260805 23:15:18.325289 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:18.357647 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:18.346823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53380 (local address 127.23.88.65:33373)
0805 23:15:18.347259 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:18.347267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:18.347335 (+ 68us) server_negotiation.cc:407] Connection header received
0805 23:15:18.347388 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:18.347395 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:18.347592 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:18.347723 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:18.348612 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.349418 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.350277 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.350681 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.350770 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.353600 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:18.353612 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:18.353627 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:18.353657 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:18.355504 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.356014 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.356020 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.356021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.356170 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.356438 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.356441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.356442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.356718 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:18.356943 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:18.357226 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:15:18.357417 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":141,"threads_started":1}
W20260805 23:15:18.358965 24069 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53380
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[23912](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:18.389891 24450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:18.381120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.23.88.65:33373)
0805 23:15:18.381307 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:18.381312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:18.381443 (+ 131us) server_negotiation.cc:407] Connection header received
0805 23:15:18.381625 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:18.381628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:18.381680 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:18.381770 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:18.382490 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.383004 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.383871 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:18.384103 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:18.384235 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:18.386687 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:18.386690 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:18.386692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:18.386720 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:18.388188 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.388653 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.388656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.388657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.388734 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:18.389011 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:18.389015 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:18.389017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:18.389195 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:18.389336 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:18.389590 (+ 254us) server_negotiation.cc:299] Negotiation successful
0805 23:15:18.389717 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:15:18.391058 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24021
I20260805 23:15:18.398032 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24167
I20260805 23:15:18.407728 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24306
I20260805 23:15:18.415521 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 23925
2026-08-05T23:15:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2675 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[24457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[24457](info): set up 2 sockets
Aug 05 23:15:18 dist-test-slave-vkjp krb5kdc[24457](info): commencing operation
krb5kdc: starting...
W20260805 23:15:19.493042 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:19 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:19.630986 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37767
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37767
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:19.749611 24470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:19.749917 24470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:19.749960 24470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:19.754200 24470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:19.754295 24470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:19.754321 24470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:19.754340 24470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:19.754405 24470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:19.760190 24470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:19.761422 24470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:19.762511 24470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:19.771607 24476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:19.771602 24475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:19.771724 24470 server_base.cc:1034] running on GCE node
W20260805 23:15:19.771602 24478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:19.772608 24470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:19.773864 24470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:19.775126 24470 hybrid_clock.cc:648] HybridClock initialized: now 1785971719775056 us; error 34 us; skew 500 ppm
Aug 05 23:15:19 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971719, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:19.778334 24470 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:19.779521 24470 webserver.cc:460] Webserver started at http://127.23.88.126:45663/ using document root <none> and password file <none>
I20260805 23:15:19.780179 24470 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:19.780287 24470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:19.780542 24470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:19.782785 24470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "02a56cb1a8594033b41feb25f103b833"
format_stamp: "Formatted at 2026-08-05 23:15:19 on dist-test-slave-vkjp"
server_key: "e2c7dbf749ebbc60b14d666b8815df8f"
server_key_iv: "24b2a0001a64f6074f9dd443704236bd"
server_key_version: "encryptionkey@0"
I20260805 23:15:19.783452 24470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "02a56cb1a8594033b41feb25f103b833"
format_stamp: "Formatted at 2026-08-05 23:15:19 on dist-test-slave-vkjp"
server_key: "e2c7dbf749ebbc60b14d666b8815df8f"
server_key_iv: "24b2a0001a64f6074f9dd443704236bd"
server_key_version: "encryptionkey@0"
I20260805 23:15:19.787556 24470 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:15:19.790133 24485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:19.791570 24470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:19.791728 24470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "02a56cb1a8594033b41feb25f103b833"
format_stamp: "Formatted at 2026-08-05 23:15:19 on dist-test-slave-vkjp"
server_key: "e2c7dbf749ebbc60b14d666b8815df8f"
server_key_iv: "24b2a0001a64f6074f9dd443704236bd"
server_key_version: "encryptionkey@0"
I20260805 23:15:19.791887 24470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:19.809505 24470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:19.813092 24470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:19.813330 24470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:19.821900 24470 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:37767
I20260805 23:15:19.821970 24540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:37767 every 8 connection(s)
I20260805 23:15:19.822981 24470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:19.826277 24541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:19.828533 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24470
I20260805 23:15:19.828656 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:19.828922 23905 external_mini_cluster.cc:1428] Setting key c8edf1dd63c1964a9b674c41a23ff5a5
I20260805 23:15:19.833107 24541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833: Bootstrap starting.
I20260805 23:15:19.835850 24541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:19.836833 24541 log.cc:824] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:15:19 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:19.839285 24541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833: No bootstrap required, opened a new log
I20260805 23:15:19.844482 24541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } }
I20260805 23:15:19.844810 24541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:19.844875 24541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02a56cb1a8594033b41feb25f103b833, State: Initialized, Role: FOLLOWER
I20260805 23:15:19.845381 24541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } }
I20260805 23:15:19.845552 24541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:19.845618 24541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:19.845713 24541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:19.846009 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:19.830673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38974 (local address 127.23.88.126:37767)
0805 23:15:19.831187 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:19.831211 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:19.831253 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:15:19.832161 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:19.832181 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:19.832554 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:19.832864 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:19.833841 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:19.834896 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:19.836131 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:19.836841 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:19.837029 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:19.839880 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:19.839887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:19.839904 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:19.839938 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:19.842392 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:19.842880 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:19.842887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:19.842894 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:19.843058 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:19.843471 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:19.843475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:19.843478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:19.844093 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:19.844298 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:19.844603 (+ 305us) server_negotiation.cc:299] Negotiation successful
0805 23:15:19.844935 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":175,"threads_started":1}
I20260805 23:15:19.846966 24541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } }
I20260805 23:15:19.847405 24541 leader_election.cc:302] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02a56cb1a8594033b41feb25f103b833; no voters:
I20260805 23:15:19.847776 24541 leader_election.cc:288] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:19.847918 24546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:19.848158 24546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [term 1 LEADER]: Becoming Leader. State: Replica: 02a56cb1a8594033b41feb25f103b833, State: Running, Role: LEADER
I20260805 23:15:19.848568 24546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } }
I20260805 23:15:19.849188 24541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:19.850445 24548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02a56cb1a8594033b41feb25f103b833. Latest consensus state: current_term: 1 leader_uuid: "02a56cb1a8594033b41feb25f103b833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } } }
I20260805 23:15:19.850585 24548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:19.851001 24547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02a56cb1a8594033b41feb25f103b833" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02a56cb1a8594033b41feb25f103b833" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37767 } } }
I20260805 23:15:19.851177 24547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:19.851468 24555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:19.855026 24555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:19.860842 24555 catalog_manager.cc:1347] Generated new cluster ID: 3034591a14224503876cf079ce2f3b6d
I20260805 23:15:19.861032 24555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:19.874454 24555 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:19.875562 24555 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:19.886415 24555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02a56cb1a8594033b41feb25f103b833: Generated new TSK 0
I20260805 23:15:19.887424 24555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:19.926610 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:20.043555 24567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:20.043876 24567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:20.043954 24567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:20.048061 24567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:20.048154 24567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:20.048242 24567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:20.053874 24567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:20.055003 24567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:20.056068 24567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:20.065840 24575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:20.065801 24573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:20.065819 24572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:20.066294 24567 server_base.cc:1034] running on GCE node
I20260805 23:15:20.066674 24567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:20.067368 24567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:20.068532 24567 hybrid_clock.cc:648] HybridClock initialized: now 1785971720068512 us; error 30 us; skew 500 ppm
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:20.071425 24567 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:20.072407 24567 webserver.cc:460] Webserver started at http://127.23.88.65:41937/ using document root <none> and password file <none>
I20260805 23:15:20.073020 24567 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:20.073117 24567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:20.073359 24567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:20.075609 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "907740f1009f4058b573ab79dad3236f"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "a990f53cdd0aa174ac781b9f046be619"
server_key_iv: "efebc1c8c7ce7102b48a62016ac6dd0d"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.076220 24567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "907740f1009f4058b573ab79dad3236f"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "a990f53cdd0aa174ac781b9f046be619"
server_key_iv: "efebc1c8c7ce7102b48a62016ac6dd0d"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.080272 24567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:20.082835 24583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.084148 24567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:15:20.084295 24567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "907740f1009f4058b573ab79dad3236f"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "a990f53cdd0aa174ac781b9f046be619"
server_key_iv: "efebc1c8c7ce7102b48a62016ac6dd0d"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.084424 24567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:20.094018 24567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:20.097312 24567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:20.097532 24567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:20.098140 24567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:20.098990 24567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:20.099062 24567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.099161 24567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:20.099218 24567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.108688 24567 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:40333
I20260805 23:15:20.108711 24700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:40333 every 8 connection(s)
I20260805 23:15:20.109854 24567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:20.113849 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24567
I20260805 23:15:20.113974 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:20.114248 23905 external_mini_cluster.cc:1428] Setting key 83badf16f7208b5e865231b52e41cc33
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:20.125070 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.111849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40949 (local address 127.23.88.126:37767)
0805 23:15:20.112032 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.112036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.112869 (+ 833us) server_negotiation.cc:407] Connection header received
0805 23:15:20.113800 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.113803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.113863 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.113979 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.115526 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.116176 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.118056 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.118304 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.118433 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.121176 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.121187 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.121189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.121215 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.122768 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.123395 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.123399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.123401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.123487 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.123844 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.123847 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.123849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.124084 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.124193 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.124725 (+ 532us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.124880 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260805 23:15:20.125908 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.112234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37767 (local address 127.23.88.65:40949)
0805 23:15:20.112702 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.112749 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.113591 (+ 842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.114201 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.114211 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.114733 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:20.115376 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.115391 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.116341 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.116348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.117912 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.117924 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.118487 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.118491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.118676 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.119394 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:20.119412 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:20.120980 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:20.122906 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.122914 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.122931 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.123283 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.123585 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.123588 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.123590 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.123739 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.124194 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:20.124201 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:20.124478 (+ 277us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.124669 (+ 191us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.125039 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:20.127327 24701 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37767
I20260805 23:15:20.127609 24701 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:20.128270 24701 heartbeater.cc:511] Master 127.23.88.126:37767 requested a full tablet report, sending...
I20260805 23:15:20.130059 24503 ts_manager.cc:194] Registered new tserver with Master: 907740f1009f4058b573ab79dad3236f (127.23.88.65:40333)
I20260805 23:15:20.131371 24503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:40949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:20.142344 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:20.271003 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:20.271423 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:20.271517 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:20.275656 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:20.275750 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:20.275843 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:20.281526 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:20.282720 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:20.283859 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:20.294082 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:20.294129 24714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:20.294129 24711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:20.294612 24706 server_base.cc:1034] running on GCE node
I20260805 23:15:20.295017 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:20.295802 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:20.297013 24706 hybrid_clock.cc:648] HybridClock initialized: now 1785971720296983 us; error 48 us; skew 500 ppm
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:20.300459 24706 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:20.301491 24706 webserver.cc:460] Webserver started at http://127.23.88.66:42671/ using document root <none> and password file <none>
I20260805 23:15:20.302114 24706 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:20.302222 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:20.302475 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:20.304683 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "4fcf788509e4458f82cf8b7488c290c1"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "0baddcfdfa39b9416fe98a50675260f8"
server_key_iv: "d0d3b4d2d098869aa93663ab10358674"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.305508 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "4fcf788509e4458f82cf8b7488c290c1"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "0baddcfdfa39b9416fe98a50675260f8"
server_key_iv: "d0d3b4d2d098869aa93663ab10358674"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.309798 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:20.312791 24722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.314236 24706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:15:20.314428 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "4fcf788509e4458f82cf8b7488c290c1"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "0baddcfdfa39b9416fe98a50675260f8"
server_key_iv: "d0d3b4d2d098869aa93663ab10358674"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.314550 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:20.333130 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:20.337426 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:20.337651 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:20.338461 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:20.339697 24706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:20.339772 24706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.339836 24706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:20.339874 24706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.353008 24706 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:33041
I20260805 23:15:20.353031 24838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:33041 every 8 connection(s)
I20260805 23:15:20.354296 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:20.359625 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24706
I20260805 23:15:20.359750 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:20.359998 23905 external_mini_cluster.cc:1428] Setting key 2187f6d7d013936b45c3a07a4d784ad2
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:20.370484 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.356214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51323 (local address 127.23.88.126:37767)
0805 23:15:20.356370 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.356376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.357258 (+ 882us) server_negotiation.cc:407] Connection header received
0805 23:15:20.358623 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.358626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.358679 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.358754 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.360534 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.361503 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.363395 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.363655 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.363717 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.366554 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.366558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.366560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.366588 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.368165 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.368805 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.368809 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.368811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.368902 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.369267 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.369271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.369273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.369489 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.369601 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.370154 (+ 553us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.370297 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:20.371767 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.356546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37767 (local address 127.23.88.66:51323)
0805 23:15:20.357061 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.357125 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.358373 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.358956 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.358968 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.359606 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:20.360346 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.360364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.361652 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.361657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.363220 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.363237 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.363895 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.363897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.364061 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.364718 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:20.364736 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:20.366342 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:20.368348 (+ 2006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.368355 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.368374 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.368678 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.369002 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.369007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.369013 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.369163 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.369601 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:20.369609 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:20.369892 (+ 283us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.370105 (+ 213us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.370476 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":102,"threads_started":1}
I20260805 23:15:20.373188 24839 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37767
I20260805 23:15:20.373451 24839 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:15:20.374099 24839 heartbeater.cc:511] Master 127.23.88.126:37767 requested a full tablet report, sending...
I20260805 23:15:20.375329 24503 ts_manager.cc:194] Registered new tserver with Master: 4fcf788509e4458f82cf8b7488c290c1 (127.23.88.66:33041)
I20260805 23:15:20.375972 24503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:51323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:20.394532 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:20.518409 24844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:20.518728 24844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:20.518807 24844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:20.522894 24844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:20.522967 24844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:20.523029 24844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:20.528395 24844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37767
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:20.529467 24844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:20.530427 24844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:20.540828 24854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:20.540828 24851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:20.541023 24844 server_base.cc:1034] running on GCE node
W20260805 23:15:20.540871 24850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:20.541690 24844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:20.542371 24844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:20.543558 24844 hybrid_clock.cc:648] HybridClock initialized: now 1785971720543521 us; error 47 us; skew 500 ppm
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:20.546315 24844 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:20.547319 24844 webserver.cc:460] Webserver started at http://127.23.88.67:40223/ using document root <none> and password file <none>
I20260805 23:15:20.547909 24844 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:20.548000 24844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:20.548235 24844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:20.550527 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "596ab9bd6e80495db6a48e55589dd0fd"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "6e25e67d692df12e79beb08e5b65b89f"
server_key_iv: "fefdfa092c7edb5c30ccdb37b7ec7bce"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.551175 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "596ab9bd6e80495db6a48e55589dd0fd"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "6e25e67d692df12e79beb08e5b65b89f"
server_key_iv: "fefdfa092c7edb5c30ccdb37b7ec7bce"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.554947 24844 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:20.557390 24862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.558652 24844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:20.558808 24844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "596ab9bd6e80495db6a48e55589dd0fd"
format_stamp: "Formatted at 2026-08-05 23:15:20 on dist-test-slave-vkjp"
server_key: "6e25e67d692df12e79beb08e5b65b89f"
server_key_iv: "fefdfa092c7edb5c30ccdb37b7ec7bce"
server_key_version: "encryptionkey@0"
I20260805 23:15:20.558929 24844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:20.569308 24844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:20.572263 24844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:20.572468 24844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:20.573062 24844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:20.573886 24844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:20.573956 24844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.574028 24844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:20.574069 24844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:20.584975 24844 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35505
I20260805 23:15:20.585000 24987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35505 every 8 connection(s)
I20260805 23:15:20.585912 24844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:20.592197 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 24844
I20260805 23:15:20.592334 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:20.592625 23905 external_mini_cluster.cc:1428] Setting key 440fcc574307db0453949aa4714f92b5
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:20.600554 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.587920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:39371 (local address 127.23.88.126:37767)
0805 23:15:20.588118 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.588123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.588978 (+ 855us) server_negotiation.cc:407] Connection header received
0805 23:15:20.589891 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.589894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.589954 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.590059 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.591246 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.591787 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.593364 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.593695 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.593797 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.596873 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.596876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.596878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.596904 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.598344 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.598861 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.598864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.598865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.598937 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.599384 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.599386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.599388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.599575 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.599653 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.600312 (+ 659us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.600417 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:15:20.601451 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.588353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37767 (local address 127.23.88.67:39371)
0805 23:15:20.588821 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.588865 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.589685 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.590237 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.590243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.590596 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:20.591093 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.591106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.591939 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.591944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.593251 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.593261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.593868 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.593871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.594132 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.594805 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:20.594829 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:20.596703 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:20.598462 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.598467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.598480 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.598765 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.599021 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:20.599024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:20.599025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:20.599295 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:20.599716 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:20.599722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:20.599970 (+ 248us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.600223 (+ 253us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.600545 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":112,"threads_started":1}
I20260805 23:15:20.602671 24988 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37767
I20260805 23:15:20.602929 24988 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:20.603523 24988 heartbeater.cc:511] Master 127.23.88.126:37767 requested a full tablet report, sending...
I20260805 23:15:20.604557 24503 ts_manager.cc:194] Registered new tserver with Master: 596ab9bd6e80495db6a48e55589dd0fd (127.23.88.67:35505)
I20260805 23:15:20.605100 24503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:39371
I20260805 23:15:20.606943 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:15:20.619331 24544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.611376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.23.88.126:37767)
0805 23:15:20.611507 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.611514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.611753 (+ 239us) server_negotiation.cc:407] Connection header received
0805 23:15:20.611918 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.611921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.611962 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.612052 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.612788 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.613303 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.614170 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.614390 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.614520 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.615950 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.615953 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.615956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.615985 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.617406 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.617938 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.617942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.617944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.618038 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.618304 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.618308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.618309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.618536 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.618647 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.618895 (+ 248us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.619120 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 23:15:20.625361 24503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:20.628856 24503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:20.645620 25000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.639290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:40333 (local address 127.0.0.1:54422)
0805 23:15:20.640598 (+ 1308us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.640613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.640740 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.641290 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.641296 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.641320 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:20.641802 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.641813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.643157 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.643160 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.644245 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.644255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.645170 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.645173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.645262 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.645302 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.645388 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.645483 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"mutex_wait_us":192,"spinlock_wait_cycles":5632,"thread_start_us":131,"threads_started":1}
I20260805 23:15:20.645690 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.639971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.23.88.65:40333)
0805 23:15:20.640454 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.640463 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.640722 (+ 259us) server_negotiation.cc:407] Connection header received
0805 23:15:20.640938 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.640942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.641105 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.641255 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:20.641922 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.642995 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.644377 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.645045 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.645164 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.645370 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.645425 (+ 55us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.645534 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":107,"threads_started":1}
I20260805 23:15:20.647272 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.639576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:33041 (local address 127.0.0.1:53982)
0805 23:15:20.640627 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.640643 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.640751 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.641472 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.641475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.641499 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:20.641802 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.641813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.643165 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.643169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.644719 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.644731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.646822 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.646827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.646937 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.646961 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.647016 (+ 55us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.647073 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":296,"spinlock_wait_cycles":24704,"thread_start_us":206,"threads_started":1}
I20260805 23:15:20.647357 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.640128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53982 (local address 127.23.88.66:33041)
0805 23:15:20.640554 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.640566 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.640855 (+ 289us) server_negotiation.cc:407] Connection header received
0805 23:15:20.640950 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.640954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.641115 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.641288 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:20.641985 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.642972 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.646118 (+ 3146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.646688 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.646817 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.646995 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.647064 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.647207 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":184,"threads_started":1}
I20260805 23:15:20.648372 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.639068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35505 (local address 127.0.0.1:57728)
0805 23:15:20.640355 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.640411 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.640617 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.641407 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.641409 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.641424 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:20.641721 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.641730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.643536 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.643540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.644718 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.644731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.647796 (+ 3065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.647800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.647894 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.647916 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.648125 (+ 209us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.648215 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1078,"mutex_wait_us":153,"thread_start_us":94,"threads_started":1}
I20260805 23:15:20.648451 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.639210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57728 (local address 127.23.88.67:35505)
0805 23:15:20.640045 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.640061 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.640501 (+ 440us) server_negotiation.cc:407] Connection header received
0805 23:15:20.640848 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.640853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.641094 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.641255 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:20.641840 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.643373 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.644865 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.645671 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.647561 (+ 1890us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.647723 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.648173 (+ 450us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.648288 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":94,"threads_started":1}
I20260805 23:15:20.649611 24632 tablet_service.cc:1467] Processing CreateTablet for tablet b6652a4e2c924baa843fe00d5ec104ee (DEFAULT_TABLE table=test-table [id=af3cde0e35a74753856c1e99bd0fdf86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:20.653085 24770 tablet_service.cc:1467] Processing CreateTablet for tablet b6652a4e2c924baa843fe00d5ec104ee (DEFAULT_TABLE table=test-table [id=af3cde0e35a74753856c1e99bd0fdf86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:20.653422 24915 tablet_service.cc:1467] Processing CreateTablet for tablet b6652a4e2c924baa843fe00d5ec104ee (DEFAULT_TABLE table=test-table [id=af3cde0e35a74753856c1e99bd0fdf86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:20.654152 24770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6652a4e2c924baa843fe00d5ec104ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:20.654327 24915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6652a4e2c924baa843fe00d5ec104ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:20.651037 24632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6652a4e2c924baa843fe00d5ec104ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:20.661502 25006 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Bootstrap starting.
I20260805 23:15:20.663247 25007 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Bootstrap starting.
I20260805 23:15:20.664352 25006 tablet_bootstrap.cc:654] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:20.665738 25006 log.cc:824] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:20.666096 25007 tablet_bootstrap.cc:654] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:20.666925 25007 log.cc:824] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:20.668049 25010 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Bootstrap starting.
I20260805 23:15:20.668346 25006 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: No bootstrap required, opened a new log
I20260805 23:15:20.668588 25006 ts_tablet_manager.cc:1397] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:15:20.669199 25007 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: No bootstrap required, opened a new log
I20260805 23:15:20.669378 25007 ts_tablet_manager.cc:1397] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:15:20.670267 25010 tablet_bootstrap.cc:654] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:20.671105 25010 log.cc:824] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:20.672281 25006 raft_consensus.cc:348] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.672559 25006 raft_consensus.cc:374] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:20.672626 25006 raft_consensus.cc:729] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 596ab9bd6e80495db6a48e55589dd0fd, State: Initialized, Role: FOLLOWER
I20260805 23:15:20.673135 25006 consensus_queue.cc:260] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.673064 25007 raft_consensus.cc:348] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.673357 25007 raft_consensus.cc:374] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:20.673413 25010 tablet_bootstrap.cc:492] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: No bootstrap required, opened a new log
I20260805 23:15:20.673413 25007 raft_consensus.cc:729] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcf788509e4458f82cf8b7488c290c1, State: Initialized, Role: FOLLOWER
I20260805 23:15:20.673584 25010 ts_tablet_manager.cc:1397] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:15:20.673841 25007 consensus_queue.cc:260] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.674314 25006 ts_tablet_manager.cc:1428] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:15:20.674829 25007 ts_tablet_manager.cc:1428] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:15:20.674826 24988 heartbeater.cc:503] Master 127.23.88.126:37767 was elected leader, sending a full tablet report...
I20260805 23:15:20.675293 24839 heartbeater.cc:503] Master 127.23.88.126:37767 was elected leader, sending a full tablet report...
I20260805 23:15:20.677739 25010 raft_consensus.cc:348] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.677950 25010 raft_consensus.cc:374] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:20.677982 25010 raft_consensus.cc:729] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907740f1009f4058b573ab79dad3236f, State: Initialized, Role: FOLLOWER
I20260805 23:15:20.678311 25010 consensus_queue.cc:260] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.678942 24701 heartbeater.cc:503] Master 127.23.88.126:37767 was elected leader, sending a full tablet report...
I20260805 23:15:20.680371 25010 ts_tablet_manager.cc:1428] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:15:20.781697 25015 raft_consensus.cc:482] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:20.781924 25015 raft_consensus.cc:504] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.783252 25015 leader_election.cc:288] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 907740f1009f4058b573ab79dad3236f (127.23.88.65:40333), 4fcf788509e4458f82cf8b7488c290c1 (127.23.88.66:33041)
I20260805 23:15:20.788890 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.783900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:36865 (local address 127.23.88.66:33041)
0805 23:15:20.784036 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.784040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.784173 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:15:20.784236 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.784239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.784289 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.784363 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:20.784705 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.785486 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.787420 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.788087 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.788221 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.788278 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.788651 (+ 373us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.788721 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:20.788913 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.783593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:40333 (local address 127.23.88.67:32813)
0805 23:15:20.783831 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.783848 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.783959 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.784284 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.784287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.784299 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:20.784588 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.784594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.785603 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.785606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.787231 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.787243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.788228 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.788232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.788438 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.788507 (+ 69us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.788607 (+ 100us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.788730 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":215}
I20260805 23:15:20.788988 25018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.783682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:33041 (local address 127.23.88.67:36865)
0805 23:15:20.784082 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:20.784094 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:20.784168 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:20.784380 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:20.784382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:20.784395 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:20.784573 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.784583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.785603 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.785606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.787233 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:20.787243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.788228 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.788232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:20.788417 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.788537 (+ 120us) client_negotiation.cc:769] Sending connection context
0805 23:15:20.788607 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:15:20.788730 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":269,"thread_start_us":136,"threads_started":1}
I20260805 23:15:20.789098 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.783686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:32813 (local address 127.23.88.65:40333)
0805 23:15:20.783835 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.783842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.783996 (+ 154us) server_negotiation.cc:407] Connection header received
0805 23:15:20.784069 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.784072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.784154 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.784247 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:20.784705 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.785486 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.787420 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.788090 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.788222 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.788278 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.788880 (+ 602us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.788974 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:15:20.789865 24790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6652a4e2c924baa843fe00d5ec104ee" candidate_uuid: "596ab9bd6e80495db6a48e55589dd0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcf788509e4458f82cf8b7488c290c1" is_pre_election: true
I20260805 23:15:20.789837 24653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6652a4e2c924baa843fe00d5ec104ee" candidate_uuid: "596ab9bd6e80495db6a48e55589dd0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907740f1009f4058b573ab79dad3236f" is_pre_election: true
I20260805 23:15:20.790308 24790 raft_consensus.cc:2454] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 596ab9bd6e80495db6a48e55589dd0fd in term 0.
I20260805 23:15:20.790967 24864 leader_election.cc:302] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fcf788509e4458f82cf8b7488c290c1, 596ab9bd6e80495db6a48e55589dd0fd; no voters:
I20260805 23:15:20.791277 25015 raft_consensus.cc:2781] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:20.791361 25015 raft_consensus.cc:482] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:20.791414 25015 raft_consensus.cc:3037] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:20.792531 25015 raft_consensus.cc:504] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.792971 25015 leader_election.cc:288] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [CANDIDATE]: Term 1 election: Requested vote from peers 907740f1009f4058b573ab79dad3236f (127.23.88.65:40333), 4fcf788509e4458f82cf8b7488c290c1 (127.23.88.66:33041)
I20260805 23:15:20.793027 24653 raft_consensus.cc:2454] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 596ab9bd6e80495db6a48e55589dd0fd in term 0.
I20260805 23:15:20.793371 24790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6652a4e2c924baa843fe00d5ec104ee" candidate_uuid: "596ab9bd6e80495db6a48e55589dd0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fcf788509e4458f82cf8b7488c290c1"
I20260805 23:15:20.793430 24652 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6652a4e2c924baa843fe00d5ec104ee" candidate_uuid: "596ab9bd6e80495db6a48e55589dd0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907740f1009f4058b573ab79dad3236f"
I20260805 23:15:20.793531 24790 raft_consensus.cc:3037] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:20.793601 24652 raft_consensus.cc:3037] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:20.794659 24790 raft_consensus.cc:2454] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 596ab9bd6e80495db6a48e55589dd0fd in term 1.
I20260805 23:15:20.794689 24652 raft_consensus.cc:2454] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 596ab9bd6e80495db6a48e55589dd0fd in term 1.
I20260805 23:15:20.795040 24866 leader_election.cc:302] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 596ab9bd6e80495db6a48e55589dd0fd, 907740f1009f4058b573ab79dad3236f; no voters:
I20260805 23:15:20.795301 25015 raft_consensus.cc:2781] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:20.795554 25015 raft_consensus.cc:686] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [term 1 LEADER]: Becoming Leader. State: Replica: 596ab9bd6e80495db6a48e55589dd0fd, State: Running, Role: LEADER
I20260805 23:15:20.795912 25015 consensus_queue.cc:237] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } }
I20260805 23:15:20.799364 24503 catalog_manager.cc:5495] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 596ab9bd6e80495db6a48e55589dd0fd (127.23.88.67). New cstate: current_term: 1 leader_uuid: "596ab9bd6e80495db6a48e55589dd0fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "596ab9bd6e80495db6a48e55589dd0fd" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35505 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "907740f1009f4058b573ab79dad3236f" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40333 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fcf788509e4458f82cf8b7488c290c1" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33041 } health_report { overall_health: UNKNOWN } } }
W20260805 23:15:20.837087 24989 tablet.cc:2369] T b6652a4e2c924baa843fe00d5ec104ee P 596ab9bd6e80495db6a48e55589dd0fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:20.850507 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.834843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.23.88.65:40333)
0805 23:15:20.835041 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.835045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.835149 (+ 104us) server_negotiation.cc:407] Connection header received
0805 23:15:20.835300 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.835303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.835348 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.835419 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.836075 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.836585 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.837724 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.837935 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.838018 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.846440 (+ 8422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.846458 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.846479 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.846520 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.848377 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.848915 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.848922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.848923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.849015 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.849321 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.849326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.849328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.849755 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.849997 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.850189 (+ 192us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.850323 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260805 23:15:20.855674 24840 tablet.cc:2369] T b6652a4e2c924baa843fe00d5ec104ee P 4fcf788509e4458f82cf8b7488c290c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:20.861276 24702 tablet.cc:2369] T b6652a4e2c924baa843fe00d5ec104ee P 907740f1009f4058b573ab79dad3236f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:20.883885 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.875141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.23.88.65:40333)
0805 23:15:20.875316 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.875320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.875422 (+ 102us) server_negotiation.cc:407] Connection header received
0805 23:15:20.875567 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.875570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.875617 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.875691 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.876418 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.876925 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.877758 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.877972 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.878051 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.880605 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.880608 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.880610 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.880637 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.882136 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.882577 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.882580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.882581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.882655 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.882902 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.882904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.882905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.883070 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.883248 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.883546 (+ 298us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.883654 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[24457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971720, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:20.912236 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:20.902940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.23.88.65:40333)
0805 23:15:20.903130 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:20.903135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:20.903314 (+ 179us) server_negotiation.cc:407] Connection header received
0805 23:15:20.903514 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:20.903516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:20.903563 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:20.903661 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:20.904402 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.904923 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.905775 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:20.906001 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:20.906113 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:20.908900 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:20.908908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:20.908910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:20.908939 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:20.910470 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.910908 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.910912 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.910914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.911004 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:20.911316 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:20.911320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:20.911322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:20.911516 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:20.911670 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:20.911933 (+ 263us) server_negotiation.cc:299] Negotiation successful
0805 23:15:20.912061 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260805 23:15:20.913983 24612 tablet_service.cc:2908] Rejecting scan request for tablet b6652a4e2c924baa843fe00d5ec104ee: Uninitialized: clean time has not yet been initialized
I20260805 23:15:20.916986 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24567
I20260805 23:15:20.924194 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24706
I20260805 23:15:20.931447 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24844
I20260805 23:15:20.937721 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 24470
2026-08-05T23:15:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2521 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[25033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[25033](info): set up 2 sockets
Aug 05 23:15:20 dist-test-slave-vkjp krb5kdc[25033](info): commencing operation
krb5kdc: starting...
W20260805 23:15:22.016717 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:22.142551 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34099
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:22.256808 25047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:22.257140 25047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:22.257304 25047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:22.261514 25047 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:22.261617 25047 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:15:22.261649 25047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:22.261696 25047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:22.261732 25047 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:22.261767 25047 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:22.267297 25047 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34099
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:22.268450 25047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:22.269541 25047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:22.278396 25055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.278389 25053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.278501 25052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:22.278697 25047 server_base.cc:1034] running on GCE node
I20260805 23:15:22.279469 25047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:22.280571 25047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:22.281764 25047 hybrid_clock.cc:648] HybridClock initialized: now 1785971722281733 us; error 42 us; skew 500 ppm
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:22.284619 25047 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:22.285586 25047 webserver.cc:460] Webserver started at http://127.23.88.126:33609/ using document root <none> and password file <none>
I20260805 23:15:22.286126 25047 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:22.286185 25047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:22.286355 25047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:22.288502 25047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "3c6aca08092846e0a0bd80657773de0a"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "1be0857268c7d928273e5996992f014d"
server_key_iv: "6b51c02242a967726255f8d22f45b9fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.289047 25047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "3c6aca08092846e0a0bd80657773de0a"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "1be0857268c7d928273e5996992f014d"
server_key_iv: "6b51c02242a967726255f8d22f45b9fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.292935 25047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:22.295413 25063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:22.296686 25047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:15:22.296826 25047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3c6aca08092846e0a0bd80657773de0a"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "1be0857268c7d928273e5996992f014d"
server_key_iv: "6b51c02242a967726255f8d22f45b9fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.296944 25047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:22.313329 25047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:22.316450 25047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:22.316677 25047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:22.325680 25047 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34099
I20260805 23:15:22.325704 25127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34099 every 8 connection(s)
I20260805 23:15:22.326674 25047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:22.329314 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25047
I20260805 23:15:22.329444 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:22.329592 25128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:22.329751 23905 external_mini_cluster.cc:1428] Setting key 31caaf5842edf3020d1473bcb3052b67
I20260805 23:15:22.336079 25128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Bootstrap starting.
I20260805 23:15:22.338567 25128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Neither blocks nor log segments found. Creating new log.
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:22.339421 25128 log.cc:824] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:22.341639 25128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: No bootstrap required, opened a new log
I20260805 23:15:22.344990 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.331303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.23.88.126:34099)
0805 23:15:22.331792 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:22.331808 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:22.331843 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:15:22.332455 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:22.332472 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:22.332796 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:22.333133 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:22.333965 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.334989 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.335918 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.336460 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.336644 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.339359 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:22.339364 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:22.339378 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:22.339407 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:22.341938 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.342494 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.342499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.342504 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.342633 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.342946 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.342948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.342949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.343426 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:22.343598 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:22.343890 (+ 292us) server_negotiation.cc:299] Negotiation successful
0805 23:15:22.344223 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":346,"thread_start_us":159,"threads_started":1}
I20260805 23:15:22.345432 25128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } }
I20260805 23:15:22.345693 25128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:22.345778 25128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c6aca08092846e0a0bd80657773de0a, State: Initialized, Role: FOLLOWER
I20260805 23:15:22.346258 25128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } }
I20260805 23:15:22.346416 25128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:22.346493 25128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:22.346590 25128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:22.347710 25128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } }
I20260805 23:15:22.348012 25128 leader_election.cc:302] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c6aca08092846e0a0bd80657773de0a; no voters:
I20260805 23:15:22.348275 25128 leader_election.cc:288] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:22.348421 25134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:22.348680 25134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [term 1 LEADER]: Becoming Leader. State: Replica: 3c6aca08092846e0a0bd80657773de0a, State: Running, Role: LEADER
I20260805 23:15:22.349057 25134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } }
I20260805 23:15:22.349644 25128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:22.351025 25136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c6aca08092846e0a0bd80657773de0a. Latest consensus state: current_term: 1 leader_uuid: "3c6aca08092846e0a0bd80657773de0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } } }
I20260805 23:15:22.351040 25135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c6aca08092846e0a0bd80657773de0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c6aca08092846e0a0bd80657773de0a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34099 } } }
I20260805 23:15:22.351255 25136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:22.351305 25135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a [sys.catalog]: This master's current role is: LEADER
W20260805 23:15:22.354781 25150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:22.354885 25150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:22.354961 25145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:22.355756 25145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:22.362177 25145 catalog_manager.cc:1347] Generated new cluster ID: a64d1773617c4029b0b3ef32ca223c4d
I20260805 23:15:22.362264 25145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:22.387044 25145 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:22.388391 25145 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:22.397610 25145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Generated new TSK 0
I20260805 23:15:22.398375 25145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:22.437922 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34099
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:22.560012 25157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:22.560297 25157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:22.560381 25157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:22.564622 25157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:22.564689 25157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:22.564759 25157 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:22.564853 25157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:22.570621 25157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:22.571802 25157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:22.572855 25157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:22.582871 25162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.582864 25165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.582864 25163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:22.583382 25157 server_base.cc:1034] running on GCE node
I20260805 23:15:22.583847 25157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:22.584558 25157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:22.585757 25157 hybrid_clock.cc:648] HybridClock initialized: now 1785971722585738 us; error 32 us; skew 500 ppm
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:22.588792 25157 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:22.589761 25157 webserver.cc:460] Webserver started at http://127.23.88.65:36919/ using document root <none> and password file <none>
I20260805 23:15:22.590355 25157 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:22.590453 25157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:22.590693 25157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:22.592880 25157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "625d0142d0d44744bd72be28bfddfdc5"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "ba32bfcab59b38288c7d2c92382cd48f"
server_key_iv: "e6b73024458d5f58a4d527d439531179"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.593484 25157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "625d0142d0d44744bd72be28bfddfdc5"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "ba32bfcab59b38288c7d2c92382cd48f"
server_key_iv: "e6b73024458d5f58a4d527d439531179"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.597539 25157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:22.600250 25172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:22.601611 25157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:22.601748 25157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "625d0142d0d44744bd72be28bfddfdc5"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "ba32bfcab59b38288c7d2c92382cd48f"
server_key_iv: "e6b73024458d5f58a4d527d439531179"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.601868 25157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:22.625362 25157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:22.629009 25157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:22.629233 25157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:22.630721 25157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:22.630797 25157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:22.630873 25157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:22.630914 25157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:22.642194 25157 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:39921
I20260805 23:15:22.642261 25290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:39921 every 8 connection(s)
I20260805 23:15:22.643438 25157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:22.645402 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.631765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:42499 (local address 127.23.88.126:34099)
0805 23:15:22.631900 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:22.631904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:22.632981 (+ 1077us) server_negotiation.cc:407] Connection header received
0805 23:15:22.634108 (+ 1127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:22.634111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:22.634176 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:22.634300 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:22.636059 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.636698 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.637898 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.638214 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.638339 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.641438 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:22.641442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:22.641444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:22.641471 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:22.643183 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.643749 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.643755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.643757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.643866 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.644307 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.644310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.644312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.644539 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:22.644635 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:22.645156 (+ 521us) server_negotiation.cc:299] Negotiation successful
0805 23:15:22.645280 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:15:22.645846 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25157
I20260805 23:15:22.645970 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:22.646252 23905 external_mini_cluster.cc:1428] Setting key 901895e09fb11202a65706b81206fea5
I20260805 23:15:22.646379 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.632317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.65:42499)
0805 23:15:22.632826 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:22.632871 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:22.633810 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:22.634542 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:22.634555 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:22.635056 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:22.635838 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.635863 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.636845 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.636848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.637761 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.637770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.638401 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.638405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.638642 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.639287 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:22.639304 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:22.641227 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:22.643297 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.643302 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.643313 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.643641 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.644000 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.644003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.644005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.644185 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.644667 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:22.644672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:22.644914 (+ 242us) client_negotiation.cc:769] Sending connection context
0805 23:15:22.645086 (+ 172us) client_negotiation.cc:240] Negotiation successful
0805 23:15:22.645341 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":152,"threads_started":1}
I20260805 23:15:22.647847 25292 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34099
I20260805 23:15:22.647995 25292 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:22.648589 25292 heartbeater.cc:511] Master 127.23.88.126:34099 requested a full tablet report, sending...
I20260805 23:15:22.650410 25081 ts_manager.cc:194] Registered new tserver with Master: 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921)
I20260805 23:15:22.651758 25081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:42499
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:22.658735 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.650441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:54365 (local address 127.23.88.126:34099)
0805 23:15:22.651238 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:22.651243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:22.651484 (+ 241us) server_negotiation.cc:407] Connection header received
0805 23:15:22.651646 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:22.651649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:22.651698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:22.651829 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:22.652289 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.652951 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.653860 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.654094 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.654206 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.655464 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:22.655468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:22.655471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:22.655505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:22.657003 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.657446 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.657450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.657452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.657539 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.657827 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.657832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.657833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.658039 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:22.658153 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:22.658423 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 23:15:22.658544 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701}
I20260805 23:15:22.658747 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.650377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.65:54365)
0805 23:15:22.651396 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:22.651409 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:22.651540 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:22.651847 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:22.651850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:22.652067 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:22.652170 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.652175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.653100 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.653104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.653734 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.653742 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.654215 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.654219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.654319 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.654892 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:22.654905 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:22.655318 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:22.657127 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.657131 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.657133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.657344 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.657637 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.657641 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.657643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.657733 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.658144 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:22.658147 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:22.658253 (+ 106us) client_negotiation.cc:769] Sending connection context
0805 23:15:22.658364 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:15:22.658541 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":945,"mutex_wait_us":2,"thread_start_us":190,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:22.677654 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34099
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:22.793491 25302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:22.793769 25302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:22.793848 25302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:22.798200 25302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:22.798278 25302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:22.798360 25302 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:22.798426 25302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:22.804694 25302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:22.805832 25302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:22.806900 25302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:22.816736 25310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.816744 25308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:22.816761 25307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:22.816954 25302 server_base.cc:1034] running on GCE node
I20260805 23:15:22.817438 25302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:22.818125 25302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:22.819286 25302 hybrid_clock.cc:648] HybridClock initialized: now 1785971722819254 us; error 43 us; skew 500 ppm
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:22.822144 25302 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:22.823156 25302 webserver.cc:460] Webserver started at http://127.23.88.66:40205/ using document root <none> and password file <none>
I20260805 23:15:22.823731 25302 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:22.823825 25302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:22.824111 25302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:22.826218 25302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "f416bc2f2f02b7aafd11767c76cf059f"
server_key_iv: "8e322c7c2d389ef6fc4f1235d559e950"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.826795 25302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "f416bc2f2f02b7aafd11767c76cf059f"
server_key_iv: "8e322c7c2d389ef6fc4f1235d559e950"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.830631 25302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:22.833199 25317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:22.834415 25302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:22.834576 25302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
format_stamp: "Formatted at 2026-08-05 23:15:22 on dist-test-slave-vkjp"
server_key: "f416bc2f2f02b7aafd11767c76cf059f"
server_key_iv: "8e322c7c2d389ef6fc4f1235d559e950"
server_key_version: "encryptionkey@0"
I20260805 23:15:22.834699 25302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:22.854724 25302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:22.858453 25302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:22.858685 25302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:22.860308 25302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:22.860388 25302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:22.860452 25302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:22.860498 25302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:22 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:22.871913 25302 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:33107
I20260805 23:15:22.871935 25436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:33107 every 8 connection(s)
I20260805 23:15:22.873134 25302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:22.874814 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25302
I20260805 23:15:22.874944 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:22.875269 23905 external_mini_cluster.cc:1428] Setting key de3c960505289d80d73b5c565ce52fb5
I20260805 23:15:22.876338 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.861279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:57297 (local address 127.23.88.126:34099)
0805 23:15:22.861462 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:22.861469 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:22.862358 (+ 889us) server_negotiation.cc:407] Connection header received
0805 23:15:22.863550 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:22.863555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:22.863625 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:22.863692 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:22.865392 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.866051 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.867803 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.868037 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.868162 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.871923 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:22.871930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:22.871937 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:22.871977 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:22.873692 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.874339 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.874342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.874344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.874430 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.874934 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.874937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.874939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.875250 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:22.875380 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:22.876043 (+ 663us) server_negotiation.cc:299] Negotiation successful
0805 23:15:22.876182 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:22.877202 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.861703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.66:57297)
0805 23:15:22.862181 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:22.862234 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:22.863282 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:22.864189 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:22.864199 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:22.864656 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:22.865231 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.865245 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.866188 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.866193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.867667 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.867678 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.868252 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.868257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.868466 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.869935 (+ 1469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:22.869964 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:22.871661 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:22.873842 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.873850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.873868 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.874199 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.874620 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.874623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.874625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.874796 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.875405 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:22.875413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:22.875733 (+ 320us) client_negotiation.cc:769] Sending connection context
0805 23:15:22.875943 (+ 210us) client_negotiation.cc:240] Negotiation successful
0805 23:15:22.876252 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":136,"threads_started":1}
I20260805 23:15:22.878530 25437 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34099
I20260805 23:15:22.878672 25437 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:22.879405 25437 heartbeater.cc:511] Master 127.23.88.126:34099 requested a full tablet report, sending...
I20260805 23:15:22.880625 25078 ts_manager.cc:194] Registered new tserver with Master: 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107)
I20260805 23:15:22.881500 25078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:57297
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:22.889341 25445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.881167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.66:51949)
0805 23:15:22.881509 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:22.881526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:22.881622 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:22.882247 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:22.882249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:22.882446 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:22.882548 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.882554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.883433 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.883435 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.884217 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:22.884232 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.884705 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.884711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:22.884835 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.885431 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:22.885447 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:22.885904 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:22.887727 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.887731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.887734 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.888021 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.888306 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:22.888309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:22.888311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:22.888402 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:22.888803 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:22.888806 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:22.888924 (+ 118us) client_negotiation.cc:769] Sending connection context
0805 23:15:22.889034 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 23:15:22.889160 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":117,"threads_started":1}
I20260805 23:15:22.889360 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:22.881837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51949 (local address 127.23.88.126:34099)
0805 23:15:22.881946 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:22.881949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:22.881960 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:15:22.881988 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:22.881990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:22.882022 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:22.882124 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:22.882667 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.883277 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.884373 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:22.884581 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:22.884664 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:22.886058 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:22.886061 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:22.886063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:22.886085 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:22.887607 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.888115 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.888117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.888119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.888197 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:22.888487 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:22.888489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:22.888490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:22.888706 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:22.888786 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:22.889122 (+ 336us) server_negotiation.cc:299] Negotiation successful
0805 23:15:22.889237 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:22.907470 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34099
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:23.027374 25447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:23.027669 25447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:23.027716 25447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:23.031790 25447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:23.031858 25447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:23.031899 25447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:23.031934 25447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:23.037273 25447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:34099
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:23.038478 25447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:23.039624 25447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:23.049364 25453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:23.049373 25455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:23.049413 25452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:23.049636 25447 server_base.cc:1034] running on GCE node
I20260805 23:15:23.050179 25447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:23.050938 25447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:23.052162 25447 hybrid_clock.cc:648] HybridClock initialized: now 1785971723052146 us; error 27 us; skew 500 ppm
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:23.055018 25447 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:23.056037 25447 webserver.cc:460] Webserver started at http://127.23.88.67:41561/ using document root <none> and password file <none>
I20260805 23:15:23.056622 25447 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:23.056686 25447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:23.056929 25447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:23.059281 25447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "66d7af8734d9405eb4effc9cdabc1599"
format_stamp: "Formatted at 2026-08-05 23:15:23 on dist-test-slave-vkjp"
server_key: "d4a8e0811d4ab91aede9e8708c6245b5"
server_key_iv: "45723feeda9f870f69e69167e201c316"
server_key_version: "encryptionkey@0"
I20260805 23:15:23.059916 25447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "66d7af8734d9405eb4effc9cdabc1599"
format_stamp: "Formatted at 2026-08-05 23:15:23 on dist-test-slave-vkjp"
server_key: "d4a8e0811d4ab91aede9e8708c6245b5"
server_key_iv: "45723feeda9f870f69e69167e201c316"
server_key_version: "encryptionkey@0"
I20260805 23:15:23.063884 25447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:23.066457 25462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:23.067790 25447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260805 23:15:23.067924 25447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "66d7af8734d9405eb4effc9cdabc1599"
format_stamp: "Formatted at 2026-08-05 23:15:23 on dist-test-slave-vkjp"
server_key: "d4a8e0811d4ab91aede9e8708c6245b5"
server_key_iv: "45723feeda9f870f69e69167e201c316"
server_key_version: "encryptionkey@0"
I20260805 23:15:23.068044 25447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:23.082454 25447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:23.085891 25447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:23.086128 25447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:23.087809 25447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:23.087888 25447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:23.087970 25447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:23.088014 25447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:23.099385 25447 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:46013
I20260805 23:15:23.099407 25581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:46013 every 8 connection(s)
I20260805 23:15:23.100461 25447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:23.103782 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.088683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:32995 (local address 127.23.88.126:34099)
0805 23:15:23.088878 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.088885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.089746 (+ 861us) server_negotiation.cc:407] Connection header received
0805 23:15:23.090867 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.090873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.090945 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.091029 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.092926 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.093833 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.095671 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.096048 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.096185 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.099687 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.099691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.099694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.099729 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.101627 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.102155 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.102157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.102159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.102238 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.102574 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.102576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.102577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.102760 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.102837 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.103476 (+ 639us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.103611 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:15:23.104700 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.089079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.67:32995)
0805 23:15:23.089565 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.089620 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.090581 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.091556 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.091566 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.091978 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:23.092698 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.092724 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.093983 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.093987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.095520 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.095531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.096213 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.096216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.096360 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.097119 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:23.097142 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:23.099433 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:23.101750 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.101755 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.101769 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.102052 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.102320 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.102322 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.102323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.102482 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.102848 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:23.102852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:23.103149 (+ 297us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.103409 (+ 260us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.103766 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":136,"threads_started":1}
I20260805 23:15:23.104772 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25447
I20260805 23:15:23.104893 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:23.105211 23905 external_mini_cluster.cc:1428] Setting key fe82caab37609330c7c3c25aa6486f9f
I20260805 23:15:23.106637 25582 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34099
I20260805 23:15:23.106781 25582 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:23.107364 25582 heartbeater.cc:511] Master 127.23.88.126:34099 requested a full tablet report, sending...
I20260805 23:15:23.108405 25080 ts_manager.cc:194] Registered new tserver with Master: 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67:46013)
I20260805 23:15:23.108951 25080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:32995
I20260805 23:15:23.115868 25588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.109277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.23.88.67:49523)
0805 23:15:23.109558 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.109573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.109693 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.109928 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.109931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.110157 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:23.110335 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.110344 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.111118 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.111121 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.111680 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.111688 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.112105 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.112107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.112174 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.112631 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:23.112642 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:23.112949 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:23.114406 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.114409 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.114411 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.114600 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.114843 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.114845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.114847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.114922 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.115326 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:23.115329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:23.115405 (+ 76us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.115547 (+ 142us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.115721 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":20,"thread_start_us":109,"threads_started":1}
I20260805 23:15:23.115968 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.109314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:49523 (local address 127.23.88.126:34099)
0805 23:15:23.109449 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.109454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.109648 (+ 194us) server_negotiation.cc:407] Connection header received
0805 23:15:23.109779 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.109781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.109820 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.109883 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.110460 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.110951 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.111783 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.112004 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.112083 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.113083 (+ 1000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.113086 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.113088 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.113110 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.114310 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.114688 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.114691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.114692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.114765 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.115007 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.115010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.115011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.115240 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.115323 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.115716 (+ 393us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.115822 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:15:23.119534 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971723, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971723, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:23.146611 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.138210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.23.88.126:34099)
0805 23:15:23.138348 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.138352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.138366 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:23.138410 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.138412 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.138467 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.138560 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.139386 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.139922 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.140727 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.140933 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.141025 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.143574 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.143578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.143580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.143606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.145005 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.145434 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.145437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.145438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.145529 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.145810 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.145813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.145814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.146008 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.146135 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.146384 (+ 249us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.146487 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:23.149943 25080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:23.153501 25080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:23.169312 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.163533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.23.88.66:33107)
0805 23:15:23.164468 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.164480 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.164503 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:15:23.164577 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.164580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.164782 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.164941 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.165201 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166363 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.167678 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.168552 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168663 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.168840 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.169029 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.169152 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":842,"thread_start_us":155,"threads_started":1}
I20260805 23:15:23.169335 25602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.162823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:33107 (local address 127.0.0.1:50394)
0805 23:15:23.164056 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.164070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.164143 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.164927 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.164929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.164974 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.165081 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.165087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166521 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.166525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.167551 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.167561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.168689 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.168819 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.168861 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.168992 (+ 131us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.169124 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1173,"mutex_wait_us":24,"thread_start_us":66,"threads_started":1}
I20260805 23:15:23.169334 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.162816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:46013 (local address 127.0.0.1:46738)
0805 23:15:23.163834 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.163892 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.164037 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.164677 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.164683 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.164710 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.164873 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.164884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166145 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.166148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.167654 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.167665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.168917 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.169039 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.169062 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.169135 (+ 73us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.169201 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260805 23:15:23.169987 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.164119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.23.88.65:39921)
0805 23:15:23.164460 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.164468 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.165005 (+ 537us) server_negotiation.cc:407] Connection header received
0805 23:15:23.165089 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.165094 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.165299 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.165449 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.165687 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166780 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168110 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.168707 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168817 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.168959 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.169439 (+ 480us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.169836 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":118,"threads_started":1}
I20260805 23:15:23.169370 25601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.163121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39921 (local address 127.0.0.1:56686)
0805 23:15:23.164525 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.164539 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.164646 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.165444 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.165447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.165467 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.165568 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.165574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166912 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.166915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.167987 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.167995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.169023 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.169026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.169122 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.169142 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.169211 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.169271 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1338,"thread_start_us":115,"threads_started":1}
I20260805 23:15:23.170377 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.163706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.23.88.67:46013)
0805 23:15:23.164115 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.164127 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.164153 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:15:23.164254 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.164260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.164502 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.164684 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.164997 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.166010 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.167795 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.168543 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.168663 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.168840 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.169949 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.170151 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":95,"threads_started":1}
I20260805 23:15:23.172789 25514 tablet_service.cc:1467] Processing CreateTablet for tablet 438908e349b64a759bce1ceeb6b19def (DEFAULT_TABLE table=test-table [id=110b8222712a4863b03d92a819adf029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.172788 25369 tablet_service.cc:1467] Processing CreateTablet for tablet 438908e349b64a759bce1ceeb6b19def (DEFAULT_TABLE table=test-table [id=110b8222712a4863b03d92a819adf029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.172784 25224 tablet_service.cc:1467] Processing CreateTablet for tablet 438908e349b64a759bce1ceeb6b19def (DEFAULT_TABLE table=test-table [id=110b8222712a4863b03d92a819adf029]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.173862 25514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 438908e349b64a759bce1ceeb6b19def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.173877 25224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 438908e349b64a759bce1ceeb6b19def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.173877 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 438908e349b64a759bce1ceeb6b19def. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.181450 25606 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Bootstrap starting.
I20260805 23:15:23.181905 25607 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Bootstrap starting.
I20260805 23:15:23.182839 25608 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Bootstrap starting.
I20260805 23:15:23.184477 25606 tablet_bootstrap.cc:654] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.184487 25607 tablet_bootstrap.cc:654] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.184640 25608 tablet_bootstrap.cc:654] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.185359 25608 log.cc:824] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:23.185406 25606 log.cc:824] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:23.185406 25607 log.cc:824] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:23.187688 25606 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: No bootstrap required, opened a new log
I20260805 23:15:23.187688 25608 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: No bootstrap required, opened a new log
I20260805 23:15:23.187909 25606 ts_tablet_manager.cc:1397] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260805 23:15:23.187911 25608 ts_tablet_manager.cc:1397] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:15:23.187963 25607 tablet_bootstrap.cc:492] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: No bootstrap required, opened a new log
I20260805 23:15:23.188184 25607 ts_tablet_manager.cc:1397] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:15:23.191792 25607 raft_consensus.cc:348] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.192063 25607 raft_consensus.cc:374] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.192130 25607 raft_consensus.cc:729] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625d0142d0d44744bd72be28bfddfdc5, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.192054 25606 raft_consensus.cc:348] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.192054 25608 raft_consensus.cc:348] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.192312 25608 raft_consensus.cc:374] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.192312 25606 raft_consensus.cc:374] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.192376 25606 raft_consensus.cc:729] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d7af8734d9405eb4effc9cdabc1599, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.192376 25608 raft_consensus.cc:729] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe8a2d62e8e424598c331b1035ee6ee, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.192700 25607 consensus_queue.cc:260] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.192842 25608 consensus_queue.cc:260] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.192859 25606 consensus_queue.cc:260] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.193892 25608 ts_tablet_manager.cc:1428] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:15:23.193892 25606 ts_tablet_manager.cc:1428] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:15:23.194046 25292 heartbeater.cc:503] Master 127.23.88.126:34099 was elected leader, sending a full tablet report...
I20260805 23:15:23.194247 25437 heartbeater.cc:503] Master 127.23.88.126:34099 was elected leader, sending a full tablet report...
I20260805 23:15:23.194264 25582 heartbeater.cc:503] Master 127.23.88.126:34099 was elected leader, sending a full tablet report...
I20260805 23:15:23.195315 25607 ts_tablet_manager.cc:1428] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:15:23.215163 25613 raft_consensus.cc:482] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:23.215413 25613 raft_consensus.cc:504] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.216743 25613 leader_election.cc:288] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107), 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921)
I20260805 23:15:23.220643 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.217139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:50869 (local address 127.23.88.66:33107)
0805 23:15:23.217275 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.217279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.217297 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:23.217488 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.217491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.217535 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.217625 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.217912 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.218439 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.219593 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.219991 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.220127 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.220187 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.220443 (+ 256us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.220529 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:15:23.220643 25471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.217043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:33107 (local address 127.23.88.67:50869)
0805 23:15:23.217219 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.217239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.217344 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.217637 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.217640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.217660 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.217787 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.217793 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.218576 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.218583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.219451 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.219460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.220127 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.220130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.220314 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.220332 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.220423 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.220500 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":51}
I20260805 23:15:23.221467 25615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.217101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39921 (local address 127.23.88.67:58643)
0805 23:15:23.217513 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.217531 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.217654 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.217928 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.217931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.217947 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.218080 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.218085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.218818 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.218823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.220291 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.220301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.221143 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.221146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.221228 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.221248 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.221310 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.221359 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":29,"thread_start_us":148,"threads_started":1}
I20260805 23:15:23.221398 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "438908e349b64a759bce1ceeb6b19def" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" is_pre_election: true
I20260805 23:15:23.221568 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.217334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:58643 (local address 127.23.88.65:39921)
0805 23:15:23.217472 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.217477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.217616 (+ 139us) server_negotiation.cc:407] Connection header received
0805 23:15:23.217774 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.217777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.217830 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.217918 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.218198 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.218696 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.220428 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.220975 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.221122 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.221171 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.221349 (+ 178us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.221441 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:23.221771 25390 raft_consensus.cc:2454] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 0.
I20260805 23:15:23.222178 25245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "438908e349b64a759bce1ceeb6b19def" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625d0142d0d44744bd72be28bfddfdc5" is_pre_election: true
I20260805 23:15:23.222328 25463 leader_election.cc:302] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66d7af8734d9405eb4effc9cdabc1599, 6fe8a2d62e8e424598c331b1035ee6ee; no voters:
I20260805 23:15:23.222517 25245 raft_consensus.cc:2454] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 0.
I20260805 23:15:23.222669 25613 raft_consensus.cc:2781] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:23.222762 25613 raft_consensus.cc:482] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:23.222846 25613 raft_consensus.cc:3037] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.224025 25613 raft_consensus.cc:504] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.224387 25613 leader_election.cc:288] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 election: Requested vote from peers 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107), 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921)
I20260805 23:15:23.224746 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "438908e349b64a759bce1ceeb6b19def" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
I20260805 23:15:23.224830 25245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "438908e349b64a759bce1ceeb6b19def" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625d0142d0d44744bd72be28bfddfdc5"
I20260805 23:15:23.224905 25390 raft_consensus.cc:3037] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.224992 25245 raft_consensus.cc:3037] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.226051 25390 raft_consensus.cc:2454] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 1.
I20260805 23:15:23.226178 25245 raft_consensus.cc:2454] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 1.
I20260805 23:15:23.226446 25463 leader_election.cc:302] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66d7af8734d9405eb4effc9cdabc1599, 6fe8a2d62e8e424598c331b1035ee6ee; no voters:
I20260805 23:15:23.226711 25613 raft_consensus.cc:2781] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:23.226998 25613 raft_consensus.cc:686] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 1 LEADER]: Becoming Leader. State: Replica: 66d7af8734d9405eb4effc9cdabc1599, State: Running, Role: LEADER
I20260805 23:15:23.227511 25613 consensus_queue.cc:237] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } }
I20260805 23:15:23.231055 25081 catalog_manager.cc:5495] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:23.260524 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.256626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46740 (local address 127.23.88.67:46013)
0805 23:15:23.256789 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.256792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.256809 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:15:23.257032 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.257034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.257091 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.257175 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:23.257417 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.257923 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.258855 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.259070 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.259168 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.259516 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.260016 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.260124 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.260246 (+ 122us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.260338 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:15:23.267655 25390 raft_consensus.cc:1261] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Refusing update from remote peer 66d7af8734d9405eb4effc9cdabc1599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.268482 25616 consensus_queue.cc:1035] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.268970 25245 raft_consensus.cc:1261] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Refusing update from remote peer 66d7af8734d9405eb4effc9cdabc1599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.269579 25616 consensus_queue.cc:1035] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.278745 25620 mvcc.cc:205] Tried to move back new op lower bound from 7315340178493173760 to 7315340178343141376. Current Snapshot: MvccSnapshot[applied={T|T < 7315340178493173760}]
I20260805 23:15:23.290983 25080 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:23.291296 25080 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:23.294168 25080 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Sending DeleteTablet for 3 replicas of tablet 438908e349b64a759bce1ceeb6b19def
I20260805 23:15:23.295469 25224 tablet_service.cc:1514] Processing DeleteTablet for tablet 438908e349b64a759bce1ceeb6b19def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:56686
I20260805 23:15:23.296150 25080 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 23:15:23.296281 25369 tablet_service.cc:1514] Processing DeleteTablet for tablet 438908e349b64a759bce1ceeb6b19def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:50394
I20260805 23:15:23.296497 25514 tablet_service.cc:1514] Processing DeleteTablet for tablet 438908e349b64a759bce1ceeb6b19def with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:46738
W20260805 23:15:23.296646 25080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:23.296877 25631 tablet_replica.cc:330] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: stopping tablet replica
I20260805 23:15:23.296988 25633 tablet_replica.cc:330] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: stopping tablet replica
I20260805 23:15:23.297415 25631 raft_consensus.cc:2229] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:23.297935 25631 raft_consensus.cc:2258] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.298246 25633 raft_consensus.cc:2229] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:23.298566 25632 tablet_replica.cc:330] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: stopping tablet replica
I20260805 23:15:23.298722 25633 raft_consensus.cc:2258] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.298878 25632 raft_consensus.cc:2229] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:23.299345 25632 raft_consensus.cc:2258] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.300366 25631 ts_tablet_manager.cc:1905] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.300899 25633 ts_tablet_manager.cc:1905] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.304152 25632 ts_tablet_manager.cc:1905] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.304270 25633 ts_tablet_manager.cc:1918] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:23.304302 25224 tablet_service.cc:1467] Processing CreateTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba (DEFAULT_TABLE table=test-table [id=750790710a3342b8bf4d116e6b8e5655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.304373 25633 log.cc:1196] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/438908e349b64a759bce1ceeb6b19def
I20260805 23:15:23.304600 25224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a83146f0b84fddb9bda95a6c3460ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.304728 25633 ts_tablet_manager.cc:1939] T 438908e349b64a759bce1ceeb6b19def P 66d7af8734d9405eb4effc9cdabc1599: Deleting consensus metadata
I20260805 23:15:23.306269 25369 tablet_service.cc:1467] Processing CreateTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba (DEFAULT_TABLE table=test-table [id=750790710a3342b8bf4d116e6b8e5655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.306531 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a83146f0b84fddb9bda95a6c3460ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.307063 25632 ts_tablet_manager.cc:1918] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:23.307230 25607 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Bootstrap starting.
I20260805 23:15:23.307968 25631 ts_tablet_manager.cc:1918] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:23.308060 25631 log.cc:1196] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/438908e349b64a759bce1ceeb6b19def
I20260805 23:15:23.308323 25607 tablet_bootstrap.cc:654] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.308379 25631 ts_tablet_manager.cc:1939] T 438908e349b64a759bce1ceeb6b19def P 625d0142d0d44744bd72be28bfddfdc5: Deleting consensus metadata
I20260805 23:15:23.309108 25608 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Bootstrap starting.
I20260805 23:15:23.309585 25632 log.cc:1196] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/438908e349b64a759bce1ceeb6b19def
I20260805 23:15:23.309677 25066 catalog_manager.cc:4841] TS 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921): tablet 438908e349b64a759bce1ceeb6b19def (table test-table [id=110b8222712a4863b03d92a819adf029]) successfully deleted
I20260805 23:15:23.309939 25632 ts_tablet_manager.cc:1939] T 438908e349b64a759bce1ceeb6b19def P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting consensus metadata
I20260805 23:15:23.310148 25608 tablet_bootstrap.cc:654] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.310578 25607 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: No bootstrap required, opened a new log
I20260805 23:15:23.310667 25607 ts_tablet_manager.cc:1397] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:23.310871 25067 catalog_manager.cc:4841] TS 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67:46013): tablet 438908e349b64a759bce1ceeb6b19def (table test-table [id=110b8222712a4863b03d92a819adf029]) successfully deleted
I20260805 23:15:23.311075 25064 catalog_manager.cc:4841] TS 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107): tablet 438908e349b64a759bce1ceeb6b19def (table test-table [id=110b8222712a4863b03d92a819adf029]) successfully deleted
I20260805 23:15:23.311545 25514 tablet_service.cc:1467] Processing CreateTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba (DEFAULT_TABLE table=test-table [id=750790710a3342b8bf4d116e6b8e5655]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:23.311596 25607 raft_consensus.cc:348] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.311717 25607 raft_consensus.cc:374] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.311749 25607 raft_consensus.cc:729] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625d0142d0d44744bd72be28bfddfdc5, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.311856 25514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56a83146f0b84fddb9bda95a6c3460ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.311893 25607 consensus_queue.cc:260] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.312193 25607 ts_tablet_manager.cc:1428] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:15:23.312822 25608 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: No bootstrap required, opened a new log
I20260805 23:15:23.312911 25608 ts_tablet_manager.cc:1397] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:23.313349 25608 raft_consensus.cc:348] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.313452 25608 raft_consensus.cc:374] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.313494 25608 raft_consensus.cc:729] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe8a2d62e8e424598c331b1035ee6ee, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.313632 25608 consensus_queue.cc:260] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.313952 25608 ts_tablet_manager.cc:1428] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:23.314503 25606 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Bootstrap starting.
I20260805 23:15:23.315572 25606 tablet_bootstrap.cc:654] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.317073 25606 tablet_bootstrap.cc:492] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: No bootstrap required, opened a new log
I20260805 23:15:23.317178 25606 ts_tablet_manager.cc:1397] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:23.317694 25606 raft_consensus.cc:348] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.317829 25606 raft_consensus.cc:374] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.317880 25606 raft_consensus.cc:729] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d7af8734d9405eb4effc9cdabc1599, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.318032 25606 consensus_queue.cc:260] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.318367 25606 ts_tablet_manager.cc:1428] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260805 23:15:23.351858 25583 tablet.cc:2369] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:23.355331 25612 raft_consensus.cc:482] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:23.355528 25612 raft_consensus.cc:504] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.356648 25612 leader_election.cc:288] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107), 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67:46013)
I20260805 23:15:23.361526 25603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.357095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:59993 (local address 127.23.88.66:33107)
0805 23:15:23.357228 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.357233 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.357597 (+ 364us) server_negotiation.cc:407] Connection header received
0805 23:15:23.357775 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.357780 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.357835 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.357906 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.358314 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.359134 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.360485 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.360888 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.361040 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.361130 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.361316 (+ 186us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.361387 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:23.361550 25638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.356909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:33107 (local address 127.23.88.65:59993)
0805 23:15:23.357485 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.357498 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.357607 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.358036 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.358038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.358053 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.358188 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.358195 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.359253 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.359257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.360362 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.360369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.361040 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.361044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.361172 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.361201 (+ 29us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.361299 (+ 98us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.361376 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":238,"thread_start_us":135,"threads_started":1}
I20260805 23:15:23.361702 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.357015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40483 (local address 127.23.88.67:46013)
0805 23:15:23.357150 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.357156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.357414 (+ 258us) server_negotiation.cc:407] Connection header received
0805 23:15:23.357687 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.357690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.357789 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.357873 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:23.358164 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.358720 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.360320 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.360959 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.361103 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.361159 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.361488 (+ 329us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.361571 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:23.362128 25637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.356906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:46013 (local address 127.23.88.65:40483)
0805 23:15:23.357323 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.357338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.357521 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.357898 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.357901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.357919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:23.358038 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.358044 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.358837 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.358842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.360168 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.360179 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.361223 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.361227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.361335 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.361360 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.361902 (+ 542us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.361981 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260805 23:15:23.362190 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a83146f0b84fddb9bda95a6c3460ba" candidate_uuid: "625d0142d0d44744bd72be28bfddfdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" is_pre_election: true
I20260805 23:15:23.362353 25390 raft_consensus.cc:2454] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625d0142d0d44744bd72be28bfddfdc5 in term 0.
I20260805 23:15:23.362771 25173 leader_election.cc:302] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625d0142d0d44744bd72be28bfddfdc5, 6fe8a2d62e8e424598c331b1035ee6ee; no voters:
I20260805 23:15:23.362975 25536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a83146f0b84fddb9bda95a6c3460ba" candidate_uuid: "625d0142d0d44744bd72be28bfddfdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d7af8734d9405eb4effc9cdabc1599" is_pre_election: true
I20260805 23:15:23.363073 25612 raft_consensus.cc:2781] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:23.363169 25612 raft_consensus.cc:482] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:23.363209 25612 raft_consensus.cc:3037] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.363271 25536 raft_consensus.cc:2454] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 625d0142d0d44744bd72be28bfddfdc5 in term 0.
I20260805 23:15:23.364521 25612 raft_consensus.cc:504] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.364936 25612 leader_election.cc:288] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [CANDIDATE]: Term 1 election: Requested vote from peers 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107), 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67:46013)
I20260805 23:15:23.365295 25536 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a83146f0b84fddb9bda95a6c3460ba" candidate_uuid: "625d0142d0d44744bd72be28bfddfdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66d7af8734d9405eb4effc9cdabc1599"
I20260805 23:15:23.365296 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56a83146f0b84fddb9bda95a6c3460ba" candidate_uuid: "625d0142d0d44744bd72be28bfddfdc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
I20260805 23:15:23.365417 25536 raft_consensus.cc:3037] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.365442 25390 raft_consensus.cc:3037] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.366657 25390 raft_consensus.cc:2454] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625d0142d0d44744bd72be28bfddfdc5 in term 1.
I20260805 23:15:23.366657 25536 raft_consensus.cc:2454] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 625d0142d0d44744bd72be28bfddfdc5 in term 1.
I20260805 23:15:23.367049 25176 leader_election.cc:302] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625d0142d0d44744bd72be28bfddfdc5, 66d7af8734d9405eb4effc9cdabc1599; no voters:
I20260805 23:15:23.367269 25612 raft_consensus.cc:2781] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:23.367517 25612 raft_consensus.cc:686] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 1 LEADER]: Becoming Leader. State: Replica: 625d0142d0d44744bd72be28bfddfdc5, State: Running, Role: LEADER
I20260805 23:15:23.367790 25612 consensus_queue.cc:237] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.370576 25080 catalog_manager.cc:5495] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65). New cstate: current_term: 1 leader_uuid: "625d0142d0d44744bd72be28bfddfdc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } health_report { overall_health: UNKNOWN } } }
W20260805 23:15:23.374423 25438 tablet.cc:2369] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:23.394915 25293 tablet.cc:2369] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:23.418304 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.409068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34099 (local address 127.0.0.1:35482)
0805 23:15:23.409396 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.409408 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.409572 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.409835 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.409838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.410253 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:23.410349 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.410356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.411164 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.411169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.411746 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.411755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.412205 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.412208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.412277 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.412971 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:23.412983 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:23.414709 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:23.416613 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.416624 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.416632 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.416874 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.417161 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:23.417166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:23.417168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:23.417254 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:23.417660 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:23.417668 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:23.417789 (+ 121us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.417908 (+ 119us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.418092 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":138,"threads_started":1}
I20260805 23:15:23.418304 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.409145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.23.88.126:34099)
0805 23:15:23.409294 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.409299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.409506 (+ 207us) server_negotiation.cc:407] Connection header received
0805 23:15:23.409677 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.409679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.409725 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.409809 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.410472 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.411010 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.411866 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.412087 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.412197 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.414866 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.414869 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.414871 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.414892 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.416489 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.416987 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.416990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.416992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.417069 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.417350 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.417353 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.417354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.417550 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.417661 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.417992 (+ 331us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.418148 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":48}
I20260805 23:15:23.423646 25080 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:35482:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:15:23.424428 25080 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:23.429735 25514 tablet_service.cc:1467] Processing CreateTablet for tablet d1a30dbc72e4477e97543c123941be4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee3de4796c624bccbeee519433d11997]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:23.429735 25369 tablet_service.cc:1467] Processing CreateTablet for tablet d1a30dbc72e4477e97543c123941be4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee3de4796c624bccbeee519433d11997]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:23.429670 25224 tablet_service.cc:1467] Processing CreateTablet for tablet d1a30dbc72e4477e97543c123941be4b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ee3de4796c624bccbeee519433d11997]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:23.430169 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a30dbc72e4477e97543c123941be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.430168 25514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a30dbc72e4477e97543c123941be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.430181 25224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d1a30dbc72e4477e97543c123941be4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:23.433488 25607 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: Bootstrap starting.
I20260805 23:15:23.434309 25607 tablet_bootstrap.cc:654] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.434630 25606 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: Bootstrap starting.
I20260805 23:15:23.435766 25606 tablet_bootstrap.cc:654] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.436386 25608 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: Bootstrap starting.
I20260805 23:15:23.437122 25606 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: No bootstrap required, opened a new log
I20260805 23:15:23.437206 25606 ts_tablet_manager.cc:1397] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:15:23.437364 25607 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: No bootstrap required, opened a new log
I20260805 23:15:23.437492 25608 tablet_bootstrap.cc:654] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:23.437500 25607 ts_tablet_manager.cc:1397] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:23.437733 25606 raft_consensus.cc:348] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.437866 25606 raft_consensus.cc:374] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.437901 25606 raft_consensus.cc:729] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66d7af8734d9405eb4effc9cdabc1599, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.438007 25607 raft_consensus.cc:348] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.438041 25606 consensus_queue.cc:260] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.438125 25607 raft_consensus.cc:374] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.438170 25607 raft_consensus.cc:729] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625d0142d0d44744bd72be28bfddfdc5, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.438293 25607 consensus_queue.cc:260] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.438347 25606 ts_tablet_manager.cc:1428] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260805 23:15:23.438591 25607 ts_tablet_manager.cc:1428] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:23.438618 25616 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.438752 25616 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.439173 25612 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.439317 25612 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.440521 25608 tablet_bootstrap.cc:492] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: No bootstrap required, opened a new log
I20260805 23:15:23.440611 25608 ts_tablet_manager.cc:1397] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:23.441139 25608 raft_consensus.cc:348] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.441268 25608 raft_consensus.cc:374] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:23.441305 25608 raft_consensus.cc:729] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe8a2d62e8e424598c331b1035ee6ee, State: Initialized, Role: FOLLOWER
I20260805 23:15:23.441820 25608 consensus_queue.cc:260] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.442104 25614 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.442267 25614 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.442536 25608 ts_tablet_manager.cc:1428] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:23.656430 25616 raft_consensus.cc:482] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:23.656615 25616 raft_consensus.cc:504] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.657131 25616 leader_election.cc:288] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921), 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107)
I20260805 23:15:23.657691 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a30dbc72e4477e97543c123941be4b" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" is_pre_election: true
I20260805 23:15:23.657670 25245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a30dbc72e4477e97543c123941be4b" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625d0142d0d44744bd72be28bfddfdc5" is_pre_election: true
I20260805 23:15:23.657841 25245 raft_consensus.cc:2454] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 0.
I20260805 23:15:23.657841 25390 raft_consensus.cc:2454] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 0.
I20260805 23:15:23.658196 25465 leader_election.cc:302] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625d0142d0d44744bd72be28bfddfdc5, 66d7af8734d9405eb4effc9cdabc1599; no voters:
I20260805 23:15:23.658399 25616 raft_consensus.cc:2781] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:23.658488 25616 raft_consensus.cc:482] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:23.658560 25616 raft_consensus.cc:3037] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.659715 25616 raft_consensus.cc:504] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.660126 25616 leader_election.cc:288] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 election: Requested vote from peers 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921), 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107)
I20260805 23:15:23.660516 25245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a30dbc72e4477e97543c123941be4b" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625d0142d0d44744bd72be28bfddfdc5"
I20260805 23:15:23.660663 25245 raft_consensus.cc:3037] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.660633 25390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d1a30dbc72e4477e97543c123941be4b" candidate_uuid: "66d7af8734d9405eb4effc9cdabc1599" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe8a2d62e8e424598c331b1035ee6ee"
I20260805 23:15:23.660781 25390 raft_consensus.cc:3037] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:23.661670 25245 raft_consensus.cc:2454] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 1.
I20260805 23:15:23.661679 25390 raft_consensus.cc:2454] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66d7af8734d9405eb4effc9cdabc1599 in term 1.
I20260805 23:15:23.662009 25465 leader_election.cc:302] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 625d0142d0d44744bd72be28bfddfdc5, 66d7af8734d9405eb4effc9cdabc1599; no voters:
I20260805 23:15:23.662211 25616 raft_consensus.cc:2781] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:23.662302 25616 raft_consensus.cc:686] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [term 1 LEADER]: Becoming Leader. State: Replica: 66d7af8734d9405eb4effc9cdabc1599, State: Running, Role: LEADER
I20260805 23:15:23.662444 25616 consensus_queue.cc:237] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } }
I20260805 23:15:23.663131 25626 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: TxnStatusTablet state changed. Reason: New leader 66d7af8734d9405eb4effc9cdabc1599. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.663242 25626 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:23.663549 25652 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:15:23.663763 25652 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:15:23.664297 25081 catalog_manager.cc:5495] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 reported cstate change: term changed from 0 to 1, leader changed from <none> to 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } health_report { overall_health: HEALTHY } } }
I20260805 23:15:23.683159 25646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.679538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:46013 (local address 127.0.0.1:46744)
0805 23:15:23.679751 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.679770 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.679887 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.680233 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.680236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.680244 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:23.680364 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.680370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.681116 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.681121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.681827 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.681837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.682321 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.682325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.682416 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.682506 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.682830 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.682850 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.682915 (+ 65us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.683004 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260805 23:15:23.683159 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.679676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.23.88.67:46013)
0805 23:15:23.679832 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.679836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.679851 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:23.680047 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.680054 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.680116 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.680205 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:23.680472 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.680998 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.681986 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.682203 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.682318 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.682664 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.682740 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.682823 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.682926 (+ 103us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.682988 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260805 23:15:23.685937 25245 raft_consensus.cc:1261] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Refusing update from remote peer 66d7af8734d9405eb4effc9cdabc1599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.686088 25390 raft_consensus.cc:1261] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Refusing update from remote peer 66d7af8734d9405eb4effc9cdabc1599: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.686501 25616 consensus_queue.cc:1035] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.686827 25626 consensus_queue.cc:1035] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.688300 25614 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: TxnStatusTablet state changed. Reason: New leader 66d7af8734d9405eb4effc9cdabc1599. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.688431 25614 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.689077 25612 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: TxnStatusTablet state changed. Reason: New leader 66d7af8734d9405eb4effc9cdabc1599. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.689208 25612 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.690493 25616 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.690610 25616 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:23.690698 25619 mvcc.cc:205] Tried to move back new op lower bound from 7315340180213137408 to 7315340180122849280. Current Snapshot: MvccSnapshot[applied={T|T < 7315340180213137408}]
I20260805 23:15:23.690927 25612 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.690986 25621 mvcc.cc:205] Tried to move back new op lower bound from 7315340180213137408 to 7315340180122849280. Current Snapshot: MvccSnapshot[applied={T|T < 7315340180213137408}]
I20260805 23:15:23.691041 25612 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 625d0142d0d44744bd72be28bfddfdc5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.695465 25613 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.695607 25613 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 66d7af8734d9405eb4effc9cdabc1599: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:23.698508 25614 tablet_replica.cc:439] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "66d7af8734d9405eb4effc9cdabc1599" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625d0142d0d44744bd72be28bfddfdc5" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39921 } } peers { permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 } } peers { permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 } } }
I20260805 23:15:23.698670 25614 tablet_replica.cc:442] T d1a30dbc72e4477e97543c123941be4b P 6fe8a2d62e8e424598c331b1035ee6ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:23.701444 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.696952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.23.88.65:39921)
0805 23:15:23.697086 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.697091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.697209 (+ 118us) server_negotiation.cc:407] Connection header received
0805 23:15:23.697383 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.697386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.697456 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.697572 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:23.697869 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.698575 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.699785 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.700100 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.700208 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.700513 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.701041 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.701173 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.701235 (+ 62us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.701293 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:15:23.712903 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.709183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:46013 (local address 127.23.88.65:33773)
0805 23:15:23.709603 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.709615 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.709748 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.710037 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.710040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.710047 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:23.710183 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.710188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.710906 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.710912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.711619 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.711628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.712106 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.712110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.712210 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.712256 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.712573 (+ 317us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.712582 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.712650 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.712732 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":120,"threads_started":1}
I20260805 23:15:23.712881 25604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.709286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33773 (local address 127.23.88.67:46013)
0805 23:15:23.709441 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.709445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.709705 (+ 260us) server_negotiation.cc:407] Connection header received
0805 23:15:23.709869 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.709872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.709943 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.710025 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:23.710295 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.710800 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.711747 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.711989 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.712102 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.712407 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.712489 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.712568 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.712676 (+ 108us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.712731 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:15:23.721262 25390 raft_consensus.cc:1261] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Refusing update from remote peer 625d0142d0d44744bd72be28bfddfdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.721558 25536 raft_consensus.cc:1261] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Refusing update from remote peer 625d0142d0d44744bd72be28bfddfdc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:23.721769 25612 consensus_queue.cc:1035] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fe8a2d62e8e424598c331b1035ee6ee" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 33107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.722218 25612 consensus_queue.cc:1035] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66d7af8734d9405eb4effc9cdabc1599" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:23.748895 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.744795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:43521 (local address 127.23.88.65:39921)
0805 23:15:23.744932 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.744936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.745190 (+ 254us) server_negotiation.cc:407] Connection header received
0805 23:15:23.745311 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.745314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.745370 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.745455 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:23.745717 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.746476 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.747650 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.747938 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.748050 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.748353 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.748472 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.748553 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.748689 (+ 136us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.748757 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:15:23.748974 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.744705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39921 (local address 127.23.88.67:43521)
0805 23:15:23.745079 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:23.745094 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:23.745222 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:23.745479 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:23.745482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:23.745490 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:23.745599 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.745604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.746595 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.746601 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.747509 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:23.747520 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.748049 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.748053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:23.748146 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.748205 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:23.748556 (+ 351us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:23.748565 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:15:23.748639 (+ 74us) client_negotiation.cc:240] Negotiation successful
0805 23:15:23.748735 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260805 23:15:23.779531 25081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:23.779769 25081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:23.782401 25081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c6aca08092846e0a0bd80657773de0a: Sending DeleteTablet for 3 replicas of tablet 56a83146f0b84fddb9bda95a6c3460ba
I20260805 23:15:23.783131 25224 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:56686
I20260805 23:15:23.783345 25631 tablet_replica.cc:330] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: stopping tablet replica
I20260805 23:15:23.783339 25369 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:50394
I20260805 23:15:23.783488 25631 raft_consensus.cc:2229] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:23.783776 25514 tablet_service.cc:1514] Processing DeleteTablet for tablet 56a83146f0b84fddb9bda95a6c3460ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:23 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:46738
I20260805 23:15:23.783846 25631 raft_consensus.cc:2258] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.783965 25633 tablet_replica.cc:330] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: stopping tablet replica
I20260805 23:15:23.784054 25632 tablet_replica.cc:330] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: stopping tablet replica
I20260805 23:15:23.784094 25633 raft_consensus.cc:2229] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:23.784188 25632 raft_consensus.cc:2229] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:23.784226 25633 raft_consensus.cc:2258] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.784319 25632 raft_consensus.cc:2258] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:23.785168 25631 ts_tablet_manager.cc:1905] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.785168 25633 ts_tablet_manager.cc:1905] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.785296 25632 ts_tablet_manager.cc:1905] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:23.787786 25632 ts_tablet_manager.cc:1918] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:23.787874 25632 log.cc:1196] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/56a83146f0b84fddb9bda95a6c3460ba
I20260805 23:15:23.788095 25633 ts_tablet_manager.cc:1918] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:23.788095 25631 ts_tablet_manager.cc:1918] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:23.788156 25632 ts_tablet_manager.cc:1939] T 56a83146f0b84fddb9bda95a6c3460ba P 6fe8a2d62e8e424598c331b1035ee6ee: Deleting consensus metadata
I20260805 23:15:23.788187 25633 log.cc:1196] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/56a83146f0b84fddb9bda95a6c3460ba
I20260805 23:15:23.788187 25631 log.cc:1196] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/56a83146f0b84fddb9bda95a6c3460ba
I20260805 23:15:23.788457 25633 ts_tablet_manager.cc:1939] T 56a83146f0b84fddb9bda95a6c3460ba P 66d7af8734d9405eb4effc9cdabc1599: Deleting consensus metadata
I20260805 23:15:23.788486 25631 ts_tablet_manager.cc:1939] T 56a83146f0b84fddb9bda95a6c3460ba P 625d0142d0d44744bd72be28bfddfdc5: Deleting consensus metadata
I20260805 23:15:23.788972 25064 catalog_manager.cc:4841] TS 6fe8a2d62e8e424598c331b1035ee6ee (127.23.88.66:33107): tablet 56a83146f0b84fddb9bda95a6c3460ba (table test-table [id=750790710a3342b8bf4d116e6b8e5655]) successfully deleted
I20260805 23:15:23.789160 25066 catalog_manager.cc:4841] TS 625d0142d0d44744bd72be28bfddfdc5 (127.23.88.65:39921): tablet 56a83146f0b84fddb9bda95a6c3460ba (table test-table [id=750790710a3342b8bf4d116e6b8e5655]) successfully deleted
I20260805 23:15:23.789160 25067 catalog_manager.cc:4841] TS 66d7af8734d9405eb4effc9cdabc1599 (127.23.88.67:46013): tablet 56a83146f0b84fddb9bda95a6c3460ba (table test-table [id=750790710a3342b8bf4d116e6b8e5655]) successfully deleted
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971723, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:23.802541 25605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.791565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56704 (local address 127.23.88.65:39921)
0805 23:15:23.791697 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.791704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.791990 (+ 286us) server_negotiation.cc:407] Connection header received
0805 23:15:23.792171 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.792175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.792230 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.792318 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.793084 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.793602 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.794491 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.794715 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.794836 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.798229 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.798242 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.798257 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.798287 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.800298 (+ 2011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.800845 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.800857 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.800859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.800969 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.801324 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.801330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.801332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.801735 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.801970 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.802208 (+ 238us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.802342 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260805 23:15:23.803421 25285 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56704
I20260805 23:15:23.813899 25132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:23.805233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35496 (local address 127.23.88.126:34099)
0805 23:15:23.805374 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:23.805379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:23.805765 (+ 386us) server_negotiation.cc:407] Connection header received
0805 23:15:23.805917 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:23.805920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:23.805979 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:23.806074 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:23.806915 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.807671 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.808555 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:23.808796 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:23.808918 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:23.810385 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:23.810389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:23.810392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:23.810422 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:23.812124 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.812582 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.812586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.812588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.812680 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:23.812971 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:23.812978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:23.812979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:23.813183 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:23.813325 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:23.813595 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 23:15:23.813728 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:15:23.814328 25081 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35496
I20260805 23:15:23.815743 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25157
I20260805 23:15:23.826071 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25302
I20260805 23:15:23.833799 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25447
I20260805 23:15:23.841737 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25047
2026-08-05T23:15:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2905 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25673](info): set up 2 sockets
Aug 05 23:15:23 dist-test-slave-vkjp krb5kdc[25673](info): commencing operation
krb5kdc: starting...
W20260805 23:15:24.908609 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:24 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:25.043908 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43993
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:43993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:25.158255 25688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:25.158591 25688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:25.158635 25688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:25.163184 25688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:25.163260 25688 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:15:25.163278 25688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:25.163297 25688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:25.163316 25688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:25.163331 25688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:25.169303 25688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:43993
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:25.170655 25688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:25.171864 25688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:25.181711 25696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:25.181803 25693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.181813 25688 server_base.cc:1034] running on GCE node
W20260805 23:15:25.181711 25694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.182780 25688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:25.183992 25688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:25.185159 25688 hybrid_clock.cc:648] HybridClock initialized: now 1785971725185135 us; error 46 us; skew 500 ppm
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:25.188186 25688 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:25.189093 25688 webserver.cc:460] Webserver started at http://127.23.88.126:35153/ using document root <none> and password file <none>
I20260805 23:15:25.189693 25688 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:25.189759 25688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:25.189951 25688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:25.192210 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "217e6ab533884f438a6e62a58cc976b1"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f83db776f4eb4bc89ad67baa90fa0bd2"
server_key_iv: "2745cba396187af18353a0065f8eeab9"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.192853 25688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "217e6ab533884f438a6e62a58cc976b1"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f83db776f4eb4bc89ad67baa90fa0bd2"
server_key_iv: "2745cba396187af18353a0065f8eeab9"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.196805 25688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:25.199395 25703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:25.200688 25688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:15:25.200832 25688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "217e6ab533884f438a6e62a58cc976b1"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f83db776f4eb4bc89ad67baa90fa0bd2"
server_key_iv: "2745cba396187af18353a0065f8eeab9"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.200955 25688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:25.220302 25688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:25.223925 25688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:25.224159 25688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:25.232398 25688 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:43993
I20260805 23:15:25.232496 25770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:43993 every 8 connection(s)
I20260805 23:15:25.233412 25688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:25.236320 25771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:25.241434 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25688
I20260805 23:15:25.241590 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:25.241868 23905 external_mini_cluster.cc:1428] Setting key d2179d5cdec161e2b0fc5180bad021f8
I20260805 23:15:25.242211 25771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Bootstrap starting.
I20260805 23:15:25.244771 25771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:25.245748 25771 log.cc:824] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:25.248243 25771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: No bootstrap required, opened a new log
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:25.253095 25771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } }
I20260805 23:15:25.253422 25771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:25.253506 25771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217e6ab533884f438a6e62a58cc976b1, State: Initialized, Role: FOLLOWER
I20260805 23:15:25.254079 25771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } }
I20260805 23:15:25.254261 25771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:25.254334 25771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:25.254436 25771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:25.255707 25771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } }
I20260805 23:15:25.256145 25771 leader_election.cc:302] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 217e6ab533884f438a6e62a58cc976b1; no voters:
I20260805 23:15:25.256544 25771 leader_election.cc:288] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:25.256680 25777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:25.256981 25777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [term 1 LEADER]: Becoming Leader. State: Replica: 217e6ab533884f438a6e62a58cc976b1, State: Running, Role: LEADER
I20260805 23:15:25.257354 25777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } }
I20260805 23:15:25.257964 25771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:25.258646 25777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 217e6ab533884f438a6e62a58cc976b1. Latest consensus state: current_term: 1 leader_uuid: "217e6ab533884f438a6e62a58cc976b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } } }
I20260805 23:15:25.258754 25777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:25.258857 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.243425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54440 (local address 127.23.88.126:43993)
0805 23:15:25.243865 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:25.243881 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:25.243916 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:15:25.244582 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:25.244599 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:25.244997 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:25.245319 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:25.246328 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.247559 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.248767 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.249351 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.249495 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.252681 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:25.252688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:25.252708 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:25.252746 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:25.254942 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.255480 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.255489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.255497 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.255666 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.256007 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.256010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.256012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.256621 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:25.256871 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:25.257367 (+ 496us) server_negotiation.cc:299] Negotiation successful
0805 23:15:25.257725 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260805 23:15:25.260962 25779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "217e6ab533884f438a6e62a58cc976b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "217e6ab533884f438a6e62a58cc976b1" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43993 } } }
I20260805 23:15:25.261409 25779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:25.261507 25783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:25.264231 25783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:25.269752 25783 catalog_manager.cc:1347] Generated new cluster ID: 5201be6d438d44509cfec040b2d8d2eb
I20260805 23:15:25.269842 25783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:25.287688 25783 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:25.288781 25783 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:25.294246 25783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Generated new TSK 0
I20260805 23:15:25.295024 25783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:25.334187 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43993
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:25.450382 25798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:25.450670 25798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:25.450721 25798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:25.454794 25798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:25.454878 25798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:25.454922 25798 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:15:25.454942 25798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:25.454975 25798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:25.460831 25798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:25.461961 25798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:25.462952 25798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:25.473166 25803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.473207 25798 server_base.cc:1034] running on GCE node
W20260805 23:15:25.473170 25805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:25.473166 25807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.473886 25798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:25.474535 25798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:25.475724 25798 hybrid_clock.cc:648] HybridClock initialized: now 1785971725475710 us; error 25 us; skew 500 ppm
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:25.478574 25798 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:25.479696 25798 webserver.cc:460] Webserver started at http://127.23.88.65:41159/ using document root <none> and password file <none>
I20260805 23:15:25.480317 25798 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:25.480413 25798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:25.480654 25798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:25.482901 25798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "d030a52b0e7146ca83143347ebc7b4d2"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "a60690117b9895ac6f9c4514b0a4e191"
server_key_iv: "ca9418b6439d97a44d3d8db8f8ed020a"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.483582 25798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "d030a52b0e7146ca83143347ebc7b4d2"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "a60690117b9895ac6f9c4514b0a4e191"
server_key_iv: "ca9418b6439d97a44d3d8db8f8ed020a"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.487670 25798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:25.490418 25814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:25.491852 25798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:25.492008 25798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "d030a52b0e7146ca83143347ebc7b4d2"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "a60690117b9895ac6f9c4514b0a4e191"
server_key_iv: "ca9418b6439d97a44d3d8db8f8ed020a"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.492156 25798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:25.517122 25798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:25.520673 25798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:25.520951 25798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:25.523068 25798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:25.523200 25798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260805 23:15:25.523272 25798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:25.523312 25798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:25.535640 25798 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:44319
I20260805 23:15:25.535704 25936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:44319 every 8 connection(s)
I20260805 23:15:25.536623 25798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:25.538398 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.523993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:34611 (local address 127.23.88.126:43993)
0805 23:15:25.524136 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:25.524140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:25.525401 (+ 1261us) server_negotiation.cc:407] Connection header received
0805 23:15:25.526522 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:25.526528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:25.526601 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:25.526669 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:25.528588 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.529393 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.530637 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.530903 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.531045 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.534263 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:25.534268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:25.534271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:25.534309 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:25.536148 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.536725 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.536731 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.536735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.536825 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.537254 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.537257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.537259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.537510 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:25.537608 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:25.538158 (+ 550us) server_negotiation.cc:299] Negotiation successful
0805 23:15:25.538272 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:15:25.539340 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.524628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.65:34611)
0805 23:15:25.525222 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:25.525273 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:25.526226 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:25.527182 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:25.527192 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:25.527741 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:25.528397 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.528420 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.529557 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.529561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.530490 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.530501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.531162 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.531171 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.531405 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.532102 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:25.532123 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:25.534041 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:25.536293 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.536299 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.536315 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.536595 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.536938 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.536941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.536943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.537125 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.537642 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:25.537648 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:25.537927 (+ 279us) client_negotiation.cc:769] Sending connection context
0805 23:15:25.538107 (+ 180us) client_negotiation.cc:240] Negotiation successful
0805 23:15:25.538380 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":177,"threads_started":1}
I20260805 23:15:25.540679 25937 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43993
I20260805 23:15:25.540845 25937 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:25.541373 25937 heartbeater.cc:511] Master 127.23.88.126:43993 requested a full tablet report, sending...
I20260805 23:15:25.541524 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25798
I20260805 23:15:25.541671 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:25.542040 23905 external_mini_cluster.cc:1428] Setting key 8c2cba3b51b2bf8645b66f3e9a8ecbbb
I20260805 23:15:25.543184 25719 ts_manager.cc:194] Registered new tserver with Master: d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319)
I20260805 23:15:25.544780 25719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:34611
I20260805 23:15:25.551605 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.543391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.65:36883)
0805 23:15:25.543669 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:25.543681 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:25.543799 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:25.544073 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:25.544075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:25.544285 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:25.544391 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.544396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.545171 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.545175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.546191 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.546205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.546789 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.546792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.546913 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.547507 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:25.547522 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:25.547866 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:25.549966 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.549969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.549971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.550180 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.550508 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.550510 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.550511 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.550588 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.551078 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:25.551127 (+ 49us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:25.551222 (+ 95us) client_negotiation.cc:769] Sending connection context
0805 23:15:25.551312 (+ 90us) client_negotiation.cc:240] Negotiation successful
0805 23:15:25.551429 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":19,"thread_start_us":115,"threads_started":1}
I20260805 23:15:25.551800 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.543333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:36883 (local address 127.23.88.126:43993)
0805 23:15:25.543455 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:25.543461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:25.543757 (+ 296us) server_negotiation.cc:407] Connection header received
0805 23:15:25.543902 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:25.543906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:25.543951 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:25.544053 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:25.544509 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.545039 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.546342 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.546651 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.546762 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.548022 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:25.548029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:25.548035 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:25.548068 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:25.549825 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.550289 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.550295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.550297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.550401 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.550685 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.550690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.550692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.550978 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:25.551113 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:25.551416 (+ 303us) server_negotiation.cc:299] Negotiation successful
0805 23:15:25.551598 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:25.575732 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43993
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:25.697782 25946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:25.698151 25946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:25.698256 25946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:25.702809 25946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:25.702919 25946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:25.702997 25946 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:15:25.703044 25946 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:25.703138 25946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:25.709069 25946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:25.710302 25946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:25.711512 25946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:25.721766 25956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:25.721794 25953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.721933 25946 server_base.cc:1034] running on GCE node
W20260805 23:15:25.721799 25954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.722604 25946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:25.723409 25946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:25.724596 25946 hybrid_clock.cc:648] HybridClock initialized: now 1785971725724580 us; error 27 us; skew 500 ppm
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:25.727715 25946 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:25.728734 25946 webserver.cc:460] Webserver started at http://127.23.88.66:34999/ using document root <none> and password file <none>
I20260805 23:15:25.729346 25946 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:25.729447 25946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:25.729693 25946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:25.731946 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "dda2f7699a194f8fbe20a4cf0298d656"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "97ece89433716e594fa651cc5bfe8b71"
server_key_iv: "98fa3b33db70a8a351ffbe21d7e40730"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.732635 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "dda2f7699a194f8fbe20a4cf0298d656"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "97ece89433716e594fa651cc5bfe8b71"
server_key_iv: "98fa3b33db70a8a351ffbe21d7e40730"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.736720 25946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:25.739403 25963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:25.740859 25946 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:15:25.741009 25946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "dda2f7699a194f8fbe20a4cf0298d656"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "97ece89433716e594fa651cc5bfe8b71"
server_key_iv: "98fa3b33db70a8a351ffbe21d7e40730"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.741138 25946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:25.753217 25946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:25.756841 25946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:25.757015 25946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:25.758662 25946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:25.758723 25946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:25.758801 25946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:25.758836 25946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:25.770778 25946 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:46145
I20260805 23:15:25.770911 26083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:46145 every 8 connection(s)
I20260805 23:15:25.771828 25946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:25.773125 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 25946
I20260805 23:15:25.773247 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:25.773537 23905 external_mini_cluster.cc:1428] Setting key bdc6c2be195b4473658c7be671d4a15b
I20260805 23:15:25.774103 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.759798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46407 (local address 127.23.88.126:43993)
0805 23:15:25.759986 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:25.759991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:25.761102 (+ 1111us) server_negotiation.cc:407] Connection header received
0805 23:15:25.762539 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:25.762544 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:25.762631 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:25.762754 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:25.764362 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.764902 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.766248 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.766669 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.766813 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.769942 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:25.769946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:25.769949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:25.769985 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:25.771686 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.772269 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.772273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.772275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.772397 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.772825 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.772828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.772830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.773109 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:25.773226 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:25.773818 (+ 592us) server_negotiation.cc:299] Negotiation successful
0805 23:15:25.773966 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:15:25.775177 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.760312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.66:46407)
0805 23:15:25.760894 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:25.760961 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:25.762264 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:25.763055 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:25.763062 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:25.763652 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:25.764192 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.764206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.765063 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.765066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.766082 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.766093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.766905 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.766910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.767185 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.767933 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:25.767950 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:25.769723 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:25.771823 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.771829 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.771843 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.772149 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.772519 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.772522 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.772525 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.772707 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.773248 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:25.773253 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:25.773548 (+ 295us) client_negotiation.cc:769] Sending connection context
0805 23:15:25.773737 (+ 189us) client_negotiation.cc:240] Negotiation successful
0805 23:15:25.774087 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":118,"threads_started":1}
I20260805 23:15:25.778012 26084 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43993
I20260805 23:15:25.778187 26084 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:25.778731 26084 heartbeater.cc:511] Master 127.23.88.126:43993 requested a full tablet report, sending...
I20260805 23:15:25.779968 25719 ts_manager.cc:194] Registered new tserver with Master: dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145)
I20260805 23:15:25.780751 25719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:46407
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:15:25.789151 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.780885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:34701 (local address 127.23.88.126:43993)
0805 23:15:25.781107 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:25.781111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:25.781125 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:25.781164 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:25.781167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:25.781209 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:25.781292 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:25.782021 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.782760 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.784110 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.784321 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.784450 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.785792 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:25.785795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:25.785797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:25.785821 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:25.787348 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.787827 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.787830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.787832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.787922 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:25.788214 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:25.788219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:25.788221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:25.788470 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:25.788586 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:25.788855 (+ 269us) server_negotiation.cc:299] Negotiation successful
0805 23:15:25.788978 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260805 23:15:25.789165 26092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:25.780557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.66:34701)
0805 23:15:25.780902 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:25.780921 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:25.781052 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:25.781347 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:25.781353 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:25.781643 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:25.781870 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.781880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.782925 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.782929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.783971 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:25.783983 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:25.784508 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:25.784515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:25.784655 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:25.785203 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:25.785216 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:25.785655 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:25.787470 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.787475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.787477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.787703 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.788018 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:25.788022 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:25.788024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:25.788117 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:25.788575 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:25.788578 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:25.788680 (+ 102us) client_negotiation.cc:769] Sending connection context
0805 23:15:25.788791 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:15:25.788965 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":2,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:25.807250 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43993
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:25.923802 26094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:25.924154 26094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:25.924250 26094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:25.928572 26094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:25.928678 26094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:25.928746 26094 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260805 23:15:25.928791 26094 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:25.928864 26094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:25.934581 26094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:43993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:25.935882 26094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:25.936986 26094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:25.947235 26100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:25.947247 26099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:25.947247 26103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:25.947678 26094 server_base.cc:1034] running on GCE node
I20260805 23:15:25.948169 26094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:25.948829 26094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:25.950042 26094 hybrid_clock.cc:648] HybridClock initialized: now 1785971725950001 us; error 51 us; skew 500 ppm
Aug 05 23:15:25 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:25.959232 26094 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:25.960502 26094 webserver.cc:460] Webserver started at http://127.23.88.67:45965/ using document root <none> and password file <none>
I20260805 23:15:25.961187 26094 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:25.961295 26094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:25.961617 26094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:25.963989 26094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "9d14ca37dd614e3a8e0f6432006dc042"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f95f37605fc22e6df742ca92c2e9fd71"
server_key_iv: "7546e8958ac0dc55c16ee4e32749650c"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.964618 26094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "9d14ca37dd614e3a8e0f6432006dc042"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f95f37605fc22e6df742ca92c2e9fd71"
server_key_iv: "7546e8958ac0dc55c16ee4e32749650c"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.969205 26094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:25.971940 26111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:25.973344 26094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:25.973503 26094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "9d14ca37dd614e3a8e0f6432006dc042"
format_stamp: "Formatted at 2026-08-05 23:15:25 on dist-test-slave-vkjp"
server_key: "f95f37605fc22e6df742ca92c2e9fd71"
server_key_iv: "7546e8958ac0dc55c16ee4e32749650c"
server_key_version: "encryptionkey@0"
I20260805 23:15:25.973635 26094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:25.999580 26094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:26.003279 26094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:26.003511 26094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:26.005268 26094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:26.005348 26094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:26.005410 26094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:26.005441 26094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:26 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:26.016812 26094 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:44637
I20260805 23:15:26.016872 26229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:44637 every 8 connection(s)
I20260805 23:15:26.018077 26094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:26.020658 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.006125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:38499 (local address 127.23.88.126:43993)
0805 23:15:26.006290 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.006296 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.007267 (+ 971us) server_negotiation.cc:407] Connection header received
0805 23:15:26.008299 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.008304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.008380 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.008506 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:26.009827 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.010747 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.012458 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.012901 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.013053 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.016715 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:26.016720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:26.016726 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:26.016763 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:26.018427 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.019167 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.019173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.019174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.019285 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.019628 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.019630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.019631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.019836 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:26.019918 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.020405 (+ 487us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.020529 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:15:26.021510 26120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.006584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.67:38499)
0805 23:15:26.007055 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.007149 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.008017 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.008715 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.008726 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.009134 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:26.009631 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.009645 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.010904 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.010909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.012303 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.012318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.013116 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.013121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.013373 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.014729 (+ 1356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:26.014753 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:26.016513 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:26.018581 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.018587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.018603 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.019013 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.019392 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.019394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.019395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.019530 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.019928 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:26.019932 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:26.020184 (+ 252us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.020336 (+ 152us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.020594 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":137,"threads_started":1}
I20260805 23:15:26.022819 26230 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43993
I20260805 23:15:26.023005 26230 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:26.023588 26230 heartbeater.cc:511] Master 127.23.88.126:43993 requested a full tablet report, sending...
I20260805 23:15:26.024362 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26094
I20260805 23:15:26.024525 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:26.024853 23905 external_mini_cluster.cc:1428] Setting key d3751d4a75e80447dd68e0b8e8c3d75b
I20260805 23:15:26.025377 25720 ts_manager.cc:194] Registered new tserver with Master: 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:26.026178 25720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:38499
I20260805 23:15:26.027041 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:15:26.035285 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.025887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:39767 (local address 127.23.88.126:43993)
0805 23:15:26.026007 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.026012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.026032 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:15:26.026076 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.026079 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.026134 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.026230 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:26.027355 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.028170 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.029010 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.029471 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.029624 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.031059 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:26.031062 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:26.031064 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:26.031159 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:26.033167 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.033698 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.033703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.033707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.033826 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.034129 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.034133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.034137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.034423 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:26.034519 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.034921 (+ 402us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.035043 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:15:26.035372 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.025474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.23.88.67:39767)
0805 23:15:26.025817 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.025833 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.025926 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.026795 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.026801 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.027053 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:26.027219 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.027228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.028329 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.028332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.028870 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.028878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.029677 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.029684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.029806 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.030361 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:26.030381 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:26.030893 (+ 512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:26.033322 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.033326 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.033333 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.033576 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.033938 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.033940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.033942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.034025 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.034554 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:26.034558 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:26.034683 (+ 125us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.034810 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.035016 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1,"thread_start_us":147,"threads_started":1}
Aug 05 23:15:26 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:15:26 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:26.058328 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.048764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.23.88.126:43993)
0805 23:15:26.048936 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.048940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.048957 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:15:26.049003 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.049005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.049049 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.049183 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:26.050033 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.050621 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.051636 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.051911 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.052035 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.054800 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:26.054804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:26.054806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:26.054838 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:26.056563 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.056993 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.056996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.056998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.057083 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.057417 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.057419 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.057420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.057636 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:26.057740 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.058092 (+ 352us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.058197 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:15:26.061744 25719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:26.064894 25719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:26.080523 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.073867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.0.0.1:39102)
0805 23:15:26.075415 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.075432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.075535 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.075983 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.075986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.076006 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.076120 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.076130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077658 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.077661 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.079125 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.079134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.079935 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.080056 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080096 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.080214 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.080354 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":185,"thread_start_us":73,"threads_started":1}
I20260805 23:15:26.080523 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.074061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.23.88.65:44319)
0805 23:15:26.074532 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.074540 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.075542 (+ 1002us) server_negotiation.cc:407] Connection header received
0805 23:15:26.075652 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.075658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.075838 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.076013 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.076250 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077490 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079244 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.079801 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079934 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080098 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.080254 (+ 156us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.080361 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":93,"threads_started":1}
I20260805 23:15:26.080812 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.074214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42596 (local address 127.23.88.67:44637)
0805 23:15:26.074753 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.074768 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.075220 (+ 452us) server_negotiation.cc:407] Connection header received
0805 23:15:26.075311 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.075316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.075571 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.075734 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.076067 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077014 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079129 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.079860 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079964 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080128 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.080584 (+ 456us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.080679 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":151,"threads_started":1}
I20260805 23:15:26.081261 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.075028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55774 (local address 127.23.88.66:46145)
0805 23:15:26.075514 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.075525 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.075549 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:15:26.075628 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.075633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.075837 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.075997 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.076387 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077511 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.078975 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.079852 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.079978 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080128 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.081001 (+ 873us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.081106 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":150,"threads_started":1}
I20260805 23:15:26.080739 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.074832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46145 (local address 127.0.0.1:55774)
0805 23:15:26.075250 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.075264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.075344 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.076147 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.076150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.076167 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.076252 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.076257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077692 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.077694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.078843 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.078853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.080199 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.080203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.080368 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080428 (+ 60us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.080504 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.080595 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":114,"thread_start_us":95,"threads_started":1}
I20260805 23:15:26.081686 26251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.074055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:44637 (local address 127.0.0.1:42596)
0805 23:15:26.074833 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.074890 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.075037 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.075733 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.075739 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.075762 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.075931 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.075941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.077129 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.077134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.078970 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.078981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.080199 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.080203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.080373 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.080429 (+ 56us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.080505 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.081035 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":584,"thread_start_us":119,"threads_started":1}
I20260805 23:15:26.084179 25866 tablet_service.cc:1467] Processing CreateTablet for tablet d46cc9f40a0e4364bbe85df27110c194 (DEFAULT_TABLE table=test-table [id=cc503d0ea6b34213be9b933005083bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.084465 26014 tablet_service.cc:1467] Processing CreateTablet for tablet d46cc9f40a0e4364bbe85df27110c194 (DEFAULT_TABLE table=test-table [id=cc503d0ea6b34213be9b933005083bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.084465 26162 tablet_service.cc:1467] Processing CreateTablet for tablet d46cc9f40a0e4364bbe85df27110c194 (DEFAULT_TABLE table=test-table [id=cc503d0ea6b34213be9b933005083bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.085337 25866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46cc9f40a0e4364bbe85df27110c194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.085469 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46cc9f40a0e4364bbe85df27110c194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.085464 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d46cc9f40a0e4364bbe85df27110c194. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.091759 26255 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Bootstrap starting.
I20260805 23:15:26.093930 26257 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Bootstrap starting.
I20260805 23:15:26.094440 26255 tablet_bootstrap.cc:654] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.095299 26255 log.cc:824] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:26.095477 26256 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Bootstrap starting.
I20260805 23:15:26.096050 26257 tablet_bootstrap.cc:654] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.096926 26257 log.cc:824] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:26.097571 26256 tablet_bootstrap.cc:654] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.097625 26255 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: No bootstrap required, opened a new log
I20260805 23:15:26.097833 26255 ts_tablet_manager.cc:1397] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260805 23:15:26.098407 26256 log.cc:824] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:26.099162 26257 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: No bootstrap required, opened a new log
I20260805 23:15:26.099370 26257 ts_tablet_manager.cc:1397] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:15:26.101729 26255 raft_consensus.cc:348] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.102020 26255 raft_consensus.cc:374] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.102080 26255 raft_consensus.cc:729] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.102228 26257 raft_consensus.cc:348] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.102484 26257 raft_consensus.cc:374] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.102564 26257 raft_consensus.cc:729] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.102576 26255 consensus_queue.cc:260] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.102983 26257 consensus_queue.cc:260] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.103272 25937 heartbeater.cc:503] Master 127.23.88.126:43993 was elected leader, sending a full tablet report...
I20260805 23:15:26.103658 26255 ts_tablet_manager.cc:1428] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260805 23:15:26.103688 26084 heartbeater.cc:503] Master 127.23.88.126:43993 was elected leader, sending a full tablet report...
I20260805 23:15:26.103986 26257 ts_tablet_manager.cc:1428] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260805 23:15:26.106103 26256 tablet_bootstrap.cc:492] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: No bootstrap required, opened a new log
I20260805 23:15:26.106346 26256 ts_tablet_manager.cc:1397] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260805 23:15:26.110096 26256 raft_consensus.cc:348] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.110340 26256 raft_consensus.cc:374] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.110414 26256 raft_consensus.cc:729] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.110841 26256 consensus_queue.cc:260] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.111567 26230 heartbeater.cc:503] Master 127.23.88.126:43993 was elected leader, sending a full tablet report...
I20260805 23:15:26.111949 26256 ts_tablet_manager.cc:1428] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260805 23:15:26.137895 26262 raft_consensus.cc:482] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:26.138109 26262 raft_consensus.cc:504] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.139664 26262 leader_election.cc:288] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637), dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145)
I20260805 23:15:26.145187 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.140056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:34221 (local address 127.23.88.67:44637)
0805 23:15:26.140253 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.140258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.140421 (+ 163us) server_negotiation.cc:407] Connection header received
0805 23:15:26.140603 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.140607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.140666 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.140745 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.141117 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.142002 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.143692 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.144318 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.144456 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.144514 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.144958 (+ 444us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.145052 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:15:26.145186 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.140733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:35089 (local address 127.23.88.66:46145)
0805 23:15:26.140888 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.140892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.140912 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:15:26.140966 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.140968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.141016 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.141126 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.141503 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.142218 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.143692 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.144335 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.144456 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.144514 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.144984 (+ 470us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.145060 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:26.145217 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.139911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:44637 (local address 127.23.88.65:34221)
0805 23:15:26.140314 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.140332 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.140476 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.140786 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.140789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.140824 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.140997 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.141003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.142117 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.142119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.143549 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.143561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.144542 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.144545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.144729 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.144823 (+ 94us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.144918 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.145020 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":152,"spinlock_wait_cycles":12032,"thread_start_us":148,"threads_started":1}
I20260805 23:15:26.145246 26266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.140394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46145 (local address 127.23.88.65:35089)
0805 23:15:26.140682 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.140700 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.140827 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.141151 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.141153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.141170 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.141311 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.141319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.142321 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.142324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.143549 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.143561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.144532 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.144536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.144699 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.144823 (+ 124us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.144918 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.145116 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":110,"spinlock_wait_cycles":10880,"thread_start_us":131,"threads_started":1}
I20260805 23:15:26.146060 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46cc9f40a0e4364bbe85df27110c194" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042" is_pre_election: true
I20260805 23:15:26.146198 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46cc9f40a0e4364bbe85df27110c194" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656" is_pre_election: true
I20260805 23:15:26.146548 26036 raft_consensus.cc:2454] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 0.
I20260805 23:15:26.146549 26183 raft_consensus.cc:2454] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 0.
I20260805 23:15:26.147169 25816 leader_election.cc:302] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, d030a52b0e7146ca83143347ebc7b4d2; no voters:
I20260805 23:15:26.147480 26262 raft_consensus.cc:2781] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:26.147578 26262 raft_consensus.cc:482] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:26.147624 26262 raft_consensus.cc:3037] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.148761 26262 raft_consensus.cc:504] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.149142 26262 leader_election.cc:288] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 election: Requested vote from peers 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637), dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145)
I20260805 23:15:26.149524 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46cc9f40a0e4364bbe85df27110c194" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042"
I20260805 23:15:26.149585 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d46cc9f40a0e4364bbe85df27110c194" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656"
I20260805 23:15:26.149675 26183 raft_consensus.cc:3037] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.149746 26036 raft_consensus.cc:3037] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.150857 26183 raft_consensus.cc:2454] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 1.
I20260805 23:15:26.151263 25816 leader_election.cc:302] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, d030a52b0e7146ca83143347ebc7b4d2; no voters:
I20260805 23:15:26.151460 26036 raft_consensus.cc:2454] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 1.
I20260805 23:15:26.151513 26262 raft_consensus.cc:2781] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:26.151819 26262 raft_consensus.cc:686] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 LEADER]: Becoming Leader. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Running, Role: LEADER
I20260805 23:15:26.152197 26262 consensus_queue.cc:237] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } }
I20260805 23:15:26.156095 25719 catalog_manager.cc:5495] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65). New cstate: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:15:26 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971725, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:26.217799 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.209009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43993 (local address 127.0.0.1:54456)
0805 23:15:26.209342 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.209354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.209497 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.209781 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.209783 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.210253 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:26.210350 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.210358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.211151 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.211155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.211713 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.211721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.212180 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.212183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.212265 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.212960 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:26.212973 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:26.214441 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:26.216216 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.216225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.216230 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.216469 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.216720 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:26.216722 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:26.216724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:26.216803 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:26.217206 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:26.217212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:26.217332 (+ 120us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.217450 (+ 118us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.217624 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":108,"threads_started":1}
I20260805 23:15:26.217816 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.209105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.23.88.126:43993)
0805 23:15:26.209259 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.209265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.209450 (+ 185us) server_negotiation.cc:407] Connection header received
0805 23:15:26.209613 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.209616 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.209662 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.209752 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:26.210496 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.211011 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.211843 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.212058 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.212171 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.214595 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:26.214598 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:26.214600 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:26.214623 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:26.216111 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.216561 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.216564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.216565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.216640 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:26.216885 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:26.216888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:26.216889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:26.217076 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:26.217160 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.217506 (+ 346us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.217701 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":58}
I20260805 23:15:26.222177 25719 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:54456:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:15:26.223014 25719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:26.228345 25866 tablet_service.cc:1467] Processing CreateTablet for tablet e9f9e2a1de7e4167ad6f7e775e64ff22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39ad5cc1070d4954afb4740f69a0bbcc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:26.228678 26014 tablet_service.cc:1467] Processing CreateTablet for tablet e9f9e2a1de7e4167ad6f7e775e64ff22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39ad5cc1070d4954afb4740f69a0bbcc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:26.228830 26162 tablet_service.cc:1467] Processing CreateTablet for tablet e9f9e2a1de7e4167ad6f7e775e64ff22 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=39ad5cc1070d4954afb4740f69a0bbcc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:26.229246 25866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f9e2a1de7e4167ad6f7e775e64ff22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.229305 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f9e2a1de7e4167ad6f7e775e64ff22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.229396 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f9e2a1de7e4167ad6f7e775e64ff22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.232345 26256 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: Bootstrap starting.
I20260805 23:15:26.232661 26257 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: Bootstrap starting.
I20260805 23:15:26.233455 26256 tablet_bootstrap.cc:654] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.233538 26255 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: Bootstrap starting.
I20260805 23:15:26.233677 26257 tablet_bootstrap.cc:654] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.234485 26255 tablet_bootstrap.cc:654] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.235049 26257 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: No bootstrap required, opened a new log
I20260805 23:15:26.235165 26257 ts_tablet_manager.cc:1397] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260805 23:15:26.235427 26256 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: No bootstrap required, opened a new log
I20260805 23:15:26.235517 26256 ts_tablet_manager.cc:1397] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260805 23:15:26.235677 26257 raft_consensus.cc:348] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.235793 26257 raft_consensus.cc:374] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.235827 26257 raft_consensus.cc:729] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.235966 26256 raft_consensus.cc:348] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.235966 26257 consensus_queue.cc:260] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.236065 26256 raft_consensus.cc:374] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.236105 26256 raft_consensus.cc:729] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.236264 26257 ts_tablet_manager.cc:1428] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:26.236218 26256 consensus_queue.cc:260] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.236493 26256 ts_tablet_manager.cc:1428] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:26.236732 26263 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.236871 26263 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.237020 26255 tablet_bootstrap.cc:492] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: No bootstrap required, opened a new log
I20260805 23:15:26.237090 26255 ts_tablet_manager.cc:1397] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:26.237069 26264 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.237210 26264 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.237527 26255 raft_consensus.cc:348] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.237632 26255 raft_consensus.cc:374] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.237660 26255 raft_consensus.cc:729] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.237766 26255 consensus_queue.cc:260] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.237993 26255 ts_tablet_manager.cc:1428] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:26.239895 26262 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.240083 26262 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260805 23:15:26.269583 26231 tablet.cc:2369] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:26.273617 26085 tablet.cc:2369] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:26.288169 25938 tablet.cc:2369] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:26.361858 26262 raft_consensus.cc:482] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:26.362025 26262 raft_consensus.cc:504] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.362510 26262 leader_election.cc:288] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:26.363188 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f9e2a1de7e4167ad6f7e775e64ff22" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656" is_pre_election: true
I20260805 23:15:26.363200 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f9e2a1de7e4167ad6f7e775e64ff22" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042" is_pre_election: true
I20260805 23:15:26.363337 26183 raft_consensus.cc:2454] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 0.
I20260805 23:15:26.363370 26036 raft_consensus.cc:2454] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 0.
I20260805 23:15:26.363730 25816 leader_election.cc:302] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d030a52b0e7146ca83143347ebc7b4d2, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:26.363945 26262 raft_consensus.cc:2781] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:26.364017 26262 raft_consensus.cc:482] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:26.364063 26262 raft_consensus.cc:3037] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.365360 26262 raft_consensus.cc:504] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.365783 26262 leader_election.cc:288] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 election: Requested vote from peers dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:26.366219 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f9e2a1de7e4167ad6f7e775e64ff22" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656"
I20260805 23:15:26.366261 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f9e2a1de7e4167ad6f7e775e64ff22" candidate_uuid: "d030a52b0e7146ca83143347ebc7b4d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042"
I20260805 23:15:26.366406 26036 raft_consensus.cc:3037] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.366411 26183 raft_consensus.cc:3037] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.367699 26036 raft_consensus.cc:2454] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 1.
I20260805 23:15:26.367985 26183 raft_consensus.cc:2454] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d030a52b0e7146ca83143347ebc7b4d2 in term 1.
I20260805 23:15:26.368049 25816 leader_election.cc:302] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d030a52b0e7146ca83143347ebc7b4d2, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:26.368237 26262 raft_consensus.cc:2781] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:26.368374 26262 raft_consensus.cc:686] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 LEADER]: Becoming Leader. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Running, Role: LEADER
I20260805 23:15:26.368606 26262 consensus_queue.cc:237] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.369827 26267 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: TxnStatusTablet state changed. Reason: New leader d030a52b0e7146ca83143347ebc7b4d2. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.369946 26267 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:26.370502 26279 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:15:26.370750 26279 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:15:26.370924 25720 catalog_manager.cc:5495] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65). New cstate: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:26.397853 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.393745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.23.88.65:44319)
0805 23:15:26.393875 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.393879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.393893 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:26.394053 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.394056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.394114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.394200 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:26.394453 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.394948 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.396222 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.396439 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.396544 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.396907 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.397451 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.397560 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.397677 (+ 117us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.397734 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:15:26.397904 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.393621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.0.0.1:39118)
0805 23:15:26.393776 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.393791 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.393895 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.394213 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.394216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.394224 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:26.394342 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.394351 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.395103 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.395107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.396082 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.396095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.396576 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.396580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.396690 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.396751 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.397560 (+ 809us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.397578 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.397656 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.397754 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260805 23:15:26.403496 26036 raft_consensus.cc:1261] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Refusing update from remote peer d030a52b0e7146ca83143347ebc7b4d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.403513 26183 raft_consensus.cc:1261] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Refusing update from remote peer d030a52b0e7146ca83143347ebc7b4d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.404330 26267 consensus_queue.cc:1035] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:26.404629 26262 consensus_queue.cc:1035] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:26.408428 26263 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: TxnStatusTablet state changed. Reason: New leader d030a52b0e7146ca83143347ebc7b4d2. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.408584 26263 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.408840 26264 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: TxnStatusTablet state changed. Reason: New leader d030a52b0e7146ca83143347ebc7b4d2. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.408955 26264 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.412453 26267 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.412595 26267 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:26.412531 26287 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.412652 26287 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P d030a52b0e7146ca83143347ebc7b4d2: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:26.412842 26282 mvcc.cc:205] Tried to move back new op lower bound from 7315340191340773376 to 7315340191207821312. Current Snapshot: MvccSnapshot[applied={T|T < 7315340191340773376}]
I20260805 23:15:26.414546 26263 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.414675 26263 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P dda2f7699a194f8fbe20a4cf0298d656: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.414746 26264 tablet_replica.cc:439] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d030a52b0e7146ca83143347ebc7b4d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } } }
I20260805 23:15:26.414856 26264 tablet_replica.cc:442] T e9f9e2a1de7e4167ad6f7e775e64ff22 P 9d14ca37dd614e3a8e0f6432006dc042: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:26.417459 26284 mvcc.cc:205] Tried to move back new op lower bound from 7315340191340773376 to 7315340191207821312. Current Snapshot: MvccSnapshot[applied={T|T < 7315340191340773376 or (T in {7315340191340773376})}]
I20260805 23:15:26.417510 26283 mvcc.cc:205] Tried to move back new op lower bound from 7315340191340773376 to 7315340191207821312. Current Snapshot: MvccSnapshot[applied={T|T < 7315340191340773376}]
I20260805 23:15:26.425259 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.421731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39132 (local address 127.23.88.65:44319)
0805 23:15:26.421899 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.421904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.421916 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:15:26.421971 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.421974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.422015 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.422116 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:26.422420 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.423186 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.424148 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.424396 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.424519 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.424799 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.424882 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.424961 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.425064 (+ 103us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.425115 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:15:26.437285 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.433495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:55161 (local address 127.23.88.65:44319)
0805 23:15:26.433622 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.433627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.433831 (+ 204us) server_negotiation.cc:407] Connection header received
0805 23:15:26.433950 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.433953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.434005 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.434067 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:26.434349 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.434831 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.436125 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.436351 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.436442 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.436828 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.436899 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.436957 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.437085 (+ 128us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.437148 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":50}
I20260805 23:15:26.437654 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.433424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.23.88.65:55161)
0805 23:15:26.433744 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.433759 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.433866 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.434100 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.434103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.434110 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:26.434249 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.434255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.434929 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.434932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.435982 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.435993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.436497 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.436501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.436603 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.436657 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.436990 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.436996 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.437065 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.437183 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":21,"thread_start_us":122,"threads_started":1}
I20260805 23:15:26.445052 26183 raft_consensus.cc:1261] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Refusing update from remote peer d030a52b0e7146ca83143347ebc7b4d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.445075 26036 raft_consensus.cc:1261] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Refusing update from remote peer d030a52b0e7146ca83143347ebc7b4d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.445499 26262 consensus_queue.cc:1035] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:26.445652 26287 consensus_queue.cc:1035] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:26.489876 25718 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:26.490096 25718 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:26.492883 25718 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Sending DeleteTablet for 3 replicas of tablet d46cc9f40a0e4364bbe85df27110c194
I20260805 23:15:26.493760 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet d46cc9f40a0e4364bbe85df27110c194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55774
I20260805 23:15:26.493741 25865 tablet_service.cc:1514] Processing DeleteTablet for tablet d46cc9f40a0e4364bbe85df27110c194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39102
I20260805 23:15:26.494513 26299 tablet_replica.cc:330] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: stopping tablet replica
I20260805 23:15:26.494455 26162 tablet_service.cc:1514] Processing DeleteTablet for tablet d46cc9f40a0e4364bbe85df27110c194 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:42596
I20260805 23:15:26.494460 25718 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:26.494935 25718 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:26.494997 26298 tablet_replica.cc:330] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: stopping tablet replica
I20260805 23:15:26.495306 26299 raft_consensus.cc:2229] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:26.495502 26300 tablet_replica.cc:330] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: stopping tablet replica
I20260805 23:15:26.495738 26299 raft_consensus.cc:2258] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:26.495802 26300 raft_consensus.cc:2229] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:26.496007 26300 raft_consensus.cc:2258] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:26.496474 26298 raft_consensus.cc:2229] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:26.496865 26298 raft_consensus.cc:2258] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:26.498425 26300 ts_tablet_manager.cc:1905] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:26.498426 26299 ts_tablet_manager.cc:1905] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:26.500926 26014 tablet_service.cc:1467] Processing CreateTablet for tablet ffc6411c446f4647a5a519ad08de64bb (DEFAULT_TABLE table=test-table [id=902be7d8ea614712a613e2bcf93e40dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.501230 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc6411c446f4647a5a519ad08de64bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.501755 26300 ts_tablet_manager.cc:1918] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:26.501860 26300 log.cc:1196] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/d46cc9f40a0e4364bbe85df27110c194
I20260805 23:15:26.502106 26299 ts_tablet_manager.cc:1918] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:26.502193 26300 ts_tablet_manager.cc:1939] T d46cc9f40a0e4364bbe85df27110c194 P 9d14ca37dd614e3a8e0f6432006dc042: Deleting consensus metadata
I20260805 23:15:26.502313 26299 log.cc:1196] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/d46cc9f40a0e4364bbe85df27110c194
I20260805 23:15:26.502717 26299 ts_tablet_manager.cc:1939] T d46cc9f40a0e4364bbe85df27110c194 P dda2f7699a194f8fbe20a4cf0298d656: Deleting consensus metadata
I20260805 23:15:26.503891 26162 tablet_service.cc:1467] Processing CreateTablet for tablet ffc6411c446f4647a5a519ad08de64bb (DEFAULT_TABLE table=test-table [id=902be7d8ea614712a613e2bcf93e40dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.504097 26257 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Bootstrap starting.
I20260805 23:15:26.504163 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc6411c446f4647a5a519ad08de64bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.504794 26298 ts_tablet_manager.cc:1905] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:26.505180 26257 tablet_bootstrap.cc:654] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.506407 26256 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Bootstrap starting.
I20260805 23:15:26.506870 25865 tablet_service.cc:1467] Processing CreateTablet for tablet ffc6411c446f4647a5a519ad08de64bb (DEFAULT_TABLE table=test-table [id=902be7d8ea614712a613e2bcf93e40dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:26.507182 25865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffc6411c446f4647a5a519ad08de64bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:26.507385 26256 tablet_bootstrap.cc:654] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.508036 26298 ts_tablet_manager.cc:1918] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:26.508152 26298 log.cc:1196] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/d46cc9f40a0e4364bbe85df27110c194
I20260805 23:15:26.508502 26298 ts_tablet_manager.cc:1939] T d46cc9f40a0e4364bbe85df27110c194 P d030a52b0e7146ca83143347ebc7b4d2: Deleting consensus metadata
I20260805 23:15:26.509527 26255 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Bootstrap starting.
I20260805 23:15:26.510483 26256 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: No bootstrap required, opened a new log
I20260805 23:15:26.510504 26255 tablet_bootstrap.cc:654] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:26.510581 26256 ts_tablet_manager.cc:1397] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:26.511305 26256 raft_consensus.cc:348] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.511435 26256 raft_consensus.cc:374] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.511469 26256 raft_consensus.cc:729] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.511420 25705 catalog_manager.cc:4841] TS 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637): tablet d46cc9f40a0e4364bbe85df27110c194 (table test-table [id=cc503d0ea6b34213be9b933005083bbe]) successfully deleted
I20260805 23:15:26.511688 26255 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: No bootstrap required, opened a new log
I20260805 23:15:26.511720 25705 catalog_manager.cc:4841] TS dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145): tablet d46cc9f40a0e4364bbe85df27110c194 (table test-table [id=cc503d0ea6b34213be9b933005083bbe]) successfully deleted
I20260805 23:15:26.511793 26255 ts_tablet_manager.cc:1397] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:26.511749 26256 consensus_queue.cc:260] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.512072 25706 catalog_manager.cc:4841] TS d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319): tablet d46cc9f40a0e4364bbe85df27110c194 (table test-table [id=cc503d0ea6b34213be9b933005083bbe]) successfully deleted
I20260805 23:15:26.512315 26255 raft_consensus.cc:348] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.512440 26255 raft_consensus.cc:374] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.512475 26255 raft_consensus.cc:729] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.512607 26255 consensus_queue.cc:260] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.512807 26257 tablet_bootstrap.cc:492] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: No bootstrap required, opened a new log
I20260805 23:15:26.512882 26255 ts_tablet_manager.cc:1428] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:26.512897 26257 ts_tablet_manager.cc:1397] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260805 23:15:26.513376 26257 raft_consensus.cc:348] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.513521 26257 raft_consensus.cc:374] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:26.513571 26257 raft_consensus.cc:729] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Initialized, Role: FOLLOWER
I20260805 23:15:26.513705 26257 consensus_queue.cc:260] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.513975 26257 ts_tablet_manager.cc:1428] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:26.514508 26256 ts_tablet_manager.cc:1428] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:26.863910 26264 raft_consensus.cc:482] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:26.864089 26264 raft_consensus.cc:504] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.865330 26264 leader_election.cc:288] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145)
I20260805 23:15:26.870361 26307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.865947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:58715 (local address 127.23.88.66:46145)
0805 23:15:26.866646 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.866666 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.866680 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:26.866734 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.866737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.866788 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.866898 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.867294 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.867984 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869197 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.869643 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869798 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.869858 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.870103 (+ 245us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.870193 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":118,"threads_started":1}
I20260805 23:15:26.870378 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.865951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:42347 (local address 127.23.88.65:44319)
0805 23:15:26.866125 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.866132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.866341 (+ 209us) server_negotiation.cc:407] Connection header received
0805 23:15:26.866445 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.866449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.866515 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.866607 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:26.867043 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.867705 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869153 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.869633 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869784 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.869851 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.870159 (+ 308us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.870254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:26.870358 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.865769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46145 (local address 127.23.88.67:58715)
0805 23:15:26.866280 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.866294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.866380 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.866991 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.866995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.867020 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.867161 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.867168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.868137 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.868140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.869001 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.869011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.869808 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.869960 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.869981 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.870047 (+ 66us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.870147 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":91,"thread_start_us":73,"threads_started":1}
I20260805 23:15:26.870368 26305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.865769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.23.88.67:42347)
0805 23:15:26.866162 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.866181 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.866341 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.866664 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.866668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.866693 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:26.866871 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.866884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.867846 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.867849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.868990 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.869002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.869785 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.869789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.869986 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.870007 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.870118 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.870196 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":73,"thread_start_us":90,"threads_started":1}
I20260805 23:15:26.871205 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc6411c446f4647a5a519ad08de64bb" candidate_uuid: "9d14ca37dd614e3a8e0f6432006dc042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656" is_pre_election: true
I20260805 23:15:26.871269 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc6411c446f4647a5a519ad08de64bb" candidate_uuid: "9d14ca37dd614e3a8e0f6432006dc042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2" is_pre_election: true
I20260805 23:15:26.871409 26036 raft_consensus.cc:2454] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d14ca37dd614e3a8e0f6432006dc042 in term 0.
I20260805 23:15:26.871594 25886 raft_consensus.cc:2454] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d14ca37dd614e3a8e0f6432006dc042 in term 0.
I20260805 23:15:26.871843 26113 leader_election.cc:302] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:26.872176 26264 raft_consensus.cc:2781] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:26.872259 26264 raft_consensus.cc:482] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:26.872323 26264 raft_consensus.cc:3037] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.873446 26264 raft_consensus.cc:504] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.873883 26264 leader_election.cc:288] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [CANDIDATE]: Term 1 election: Requested vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145)
I20260805 23:15:26.874312 26036 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc6411c446f4647a5a519ad08de64bb" candidate_uuid: "9d14ca37dd614e3a8e0f6432006dc042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dda2f7699a194f8fbe20a4cf0298d656"
I20260805 23:15:26.874526 26036 raft_consensus.cc:3037] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.875605 26036 raft_consensus.cc:2454] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d14ca37dd614e3a8e0f6432006dc042 in term 1.
I20260805 23:15:26.875942 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffc6411c446f4647a5a519ad08de64bb" candidate_uuid: "9d14ca37dd614e3a8e0f6432006dc042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2"
I20260805 23:15:26.875976 26113 leader_election.cc:302] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:26.876106 25886 raft_consensus.cc:3037] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:26.876158 26264 raft_consensus.cc:2781] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:26.876418 26264 raft_consensus.cc:686] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 LEADER]: Becoming Leader. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Running, Role: LEADER
I20260805 23:15:26.876690 26264 consensus_queue.cc:237] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:26.877439 25886 raft_consensus.cc:2454] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d14ca37dd614e3a8e0f6432006dc042 in term 1.
I20260805 23:15:26.879492 25720 catalog_manager.cc:5495] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "9d14ca37dd614e3a8e0f6432006dc042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } health_report { overall_health: HEALTHY } } }
I20260805 23:15:26.979699 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.975394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.23.88.67:44637)
0805 23:15:26.975673 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.975681 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.975695 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:26.975786 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.975787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.975843 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.975943 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:26.976154 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.976678 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.977767 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.978032 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.978170 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.978508 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.979168 (+ 660us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.979309 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.979429 (+ 120us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.979492 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260805 23:15:26.992026 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.988739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:49565 (local address 127.23.88.65:44319)
0805 23:15:26.988898 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:26.988902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:26.989044 (+ 142us) server_negotiation.cc:407] Connection header received
0805 23:15:26.989168 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:26.989171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:26.989218 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:26.989292 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:26.989531 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.990049 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.990998 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.991292 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.991374 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.991672 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.991746 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.991798 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:26.991855 (+ 57us) server_negotiation.cc:299] Negotiation successful
0805 23:15:26.991905 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:15:26.992064 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:26.988628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.23.88.67:49565)
0805 23:15:26.988956 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:26.988975 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:26.989078 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:26.989314 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:26.989316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:26.989322 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:26.989437 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.989441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.990180 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.990183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.990879 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:26.990890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:26.991407 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:26.991410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:26.991491 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:26.991543 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:26.991801 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:26.991804 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:15:26.991849 (+ 45us) client_negotiation.cc:240] Negotiation successful
0805 23:15:26.991914 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":123,"threads_started":1}
I20260805 23:15:26.998646 26036 raft_consensus.cc:1261] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Refusing update from remote peer 9d14ca37dd614e3a8e0f6432006dc042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.998840 25886 raft_consensus.cc:1261] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Refusing update from remote peer 9d14ca37dd614e3a8e0f6432006dc042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:26.999195 26308 consensus_queue.cc:1035] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:26.999380 26264 consensus_queue.cc:1035] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:27.020004 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:27.016037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:48155 (local address 127.23.88.67:44637)
0805 23:15:27.016144 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:27.016148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:27.016221 (+ 73us) server_negotiation.cc:407] Connection header received
0805 23:15:27.016372 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:27.016374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:27.016423 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:27.016540 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:27.016917 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:27.017459 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:27.018628 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:27.018980 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:27.019068 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:27.019504 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:27.019593 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:27.019649 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:27.019817 (+ 168us) server_negotiation.cc:299] Negotiation successful
0805 23:15:27.019872 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:15:27.020058 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:27.015876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:44637 (local address 127.23.88.65:48155)
0805 23:15:27.016154 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:27.016170 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:27.016270 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:27.016633 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:27.016636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:27.016645 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:27.016782 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:27.016789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:27.017572 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:27.017576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:27.018484 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:27.018495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:27.019202 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:27.019205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:27.019317 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:27.019356 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:27.019721 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:27.019726 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:15:27.019802 (+ 76us) client_negotiation.cc:240] Negotiation successful
0805 23:15:27.019907 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
W20260805 23:15:27.020972 26283 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 23:15:28.595443 25720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:28.595643 25720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:28.599064 25720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Sending DeleteTablet for 3 replicas of tablet ffc6411c446f4647a5a519ad08de64bb
I20260805 23:15:28.599925 25865 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc6411c446f4647a5a519ad08de64bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39102
I20260805 23:15:28.600188 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc6411c446f4647a5a519ad08de64bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55774
I20260805 23:15:28.600337 26162 tablet_service.cc:1514] Processing DeleteTablet for tablet ffc6411c446f4647a5a519ad08de64bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:42596
I20260805 23:15:28.600764 26344 tablet_replica.cc:330] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: stopping tablet replica
I20260805 23:15:28.600970 26344 raft_consensus.cc:2229] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:28.601015 26346 tablet_replica.cc:330] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: stopping tablet replica
I20260805 23:15:28.601187 26346 raft_consensus.cc:2229] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:28.601244 26344 raft_consensus.cc:2258] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.601377 26345 tablet_replica.cc:330] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: stopping tablet replica
I20260805 23:15:28.602037 26346 raft_consensus.cc:2258] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.602239 26344 ts_tablet_manager.cc:1905] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.602478 26345 raft_consensus.cc:2229] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:28.602818 26345 raft_consensus.cc:2258] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.603286 26346 ts_tablet_manager.cc:1905] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.605451 26345 ts_tablet_manager.cc:1905] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.607683 26346 ts_tablet_manager.cc:1918] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:15:28.607847 26346 log.cc:1196] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/ffc6411c446f4647a5a519ad08de64bb
I20260805 23:15:28.608271 26346 ts_tablet_manager.cc:1939] T ffc6411c446f4647a5a519ad08de64bb P 9d14ca37dd614e3a8e0f6432006dc042: Deleting consensus metadata
I20260805 23:15:28.609365 25705 catalog_manager.cc:4841] TS 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637): tablet ffc6411c446f4647a5a519ad08de64bb (table test-table [id=902be7d8ea614712a613e2bcf93e40dd]) successfully deleted
I20260805 23:15:28.609519 26344 ts_tablet_manager.cc:1918] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:15:28.609746 26344 log.cc:1196] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/ffc6411c446f4647a5a519ad08de64bb
I20260805 23:15:28.610111 26344 ts_tablet_manager.cc:1939] T ffc6411c446f4647a5a519ad08de64bb P d030a52b0e7146ca83143347ebc7b4d2: Deleting consensus metadata
I20260805 23:15:28.611027 26345 ts_tablet_manager.cc:1918] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:15:28.611169 26345 log.cc:1196] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/ffc6411c446f4647a5a519ad08de64bb
I20260805 23:15:28.611472 26345 ts_tablet_manager.cc:1939] T ffc6411c446f4647a5a519ad08de64bb P dda2f7699a194f8fbe20a4cf0298d656: Deleting consensus metadata
I20260805 23:15:28.611935 25706 catalog_manager.cc:4841] TS d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319): tablet ffc6411c446f4647a5a519ad08de64bb (table test-table [id=902be7d8ea614712a613e2bcf93e40dd]) successfully deleted
I20260805 23:15:28.612984 25705 catalog_manager.cc:4841] TS dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145): tablet ffc6411c446f4647a5a519ad08de64bb (table test-table [id=902be7d8ea614712a613e2bcf93e40dd]) successfully deleted
Aug 05 23:15:28 dist-test-slave-vkjp krb5kdc[25673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:15:28 dist-test-slave-vkjp krb5kdc[25673](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971728, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:28.638177 26354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.626450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54466 (local address 127.23.88.126:43993)
0805 23:15:28.626806 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.626817 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.626841 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:15:28.627277 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.627281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.627342 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.627476 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:28.628368 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.628989 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.629934 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.630185 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.630313 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.633328 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:28.633332 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:28.633334 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:28.633374 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:28.635267 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:28.635981 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:28.635984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:28.635986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:28.636106 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:28.636546 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:28.636552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:28.636553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:28.636928 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:28.637170 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.637778 (+ 608us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.637982 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":128,"threads_started":1}
I20260805 23:15:28.643999 25720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:28.645076 25720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:28.652648 25865 tablet_service.cc:1467] Processing CreateTablet for tablet 65ae7814b83245c486339b491f4ed21a (DEFAULT_TABLE table=test-table [id=d90c8fd2107540b3aba3dd96fe006be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.653023 25865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ae7814b83245c486339b491f4ed21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.653126 26162 tablet_service.cc:1467] Processing CreateTablet for tablet 65ae7814b83245c486339b491f4ed21a (DEFAULT_TABLE table=test-table [id=d90c8fd2107540b3aba3dd96fe006be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.653363 26014 tablet_service.cc:1467] Processing CreateTablet for tablet 65ae7814b83245c486339b491f4ed21a (DEFAULT_TABLE table=test-table [id=d90c8fd2107540b3aba3dd96fe006be7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.653473 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ae7814b83245c486339b491f4ed21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.653653 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65ae7814b83245c486339b491f4ed21a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.656282 26355 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Bootstrap starting.
I20260805 23:15:28.656845 26356 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Bootstrap starting.
I20260805 23:15:28.656872 26357 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Bootstrap starting.
I20260805 23:15:28.657655 26355 tablet_bootstrap.cc:654] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.658070 26356 tablet_bootstrap.cc:654] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.658216 26357 tablet_bootstrap.cc:654] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.660084 26357 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: No bootstrap required, opened a new log
I20260805 23:15:28.660108 26355 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: No bootstrap required, opened a new log
I20260805 23:15:28.660221 26355 ts_tablet_manager.cc:1397] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:28.660241 26357 ts_tablet_manager.cc:1397] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:28.660310 26356 tablet_bootstrap.cc:492] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: No bootstrap required, opened a new log
I20260805 23:15:28.660408 26356 ts_tablet_manager.cc:1397] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:28.660847 26357 raft_consensus.cc:348] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.660847 26356 raft_consensus.cc:348] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.660835 26355 raft_consensus.cc:348] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.660962 26357 raft_consensus.cc:374] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.660965 26356 raft_consensus.cc:374] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.660966 26355 raft_consensus.cc:374] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.661008 26357 raft_consensus.cc:729] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.661019 26356 raft_consensus.cc:729] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.661019 26355 raft_consensus.cc:729] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.661168 26356 consensus_queue.cc:260] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.661168 26355 consensus_queue.cc:260] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.661180 26357 consensus_queue.cc:260] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.661473 26356 ts_tablet_manager.cc:1428] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:15:28.661473 26355 ts_tablet_manager.cc:1428] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260805 23:15:28.661931 26357 ts_tablet_manager.cc:1428] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260805 23:15:28.702785 26361 raft_consensus.cc:482] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:28.702980 26361 raft_consensus.cc:504] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.704437 26361 leader_election.cc:288] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:28.710462 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.704877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:48689 (local address 127.23.88.65:44319)
0805 23:15:28.705342 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.705353 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.705375 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:15:28.705578 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.705585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.705681 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.705767 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:28.706205 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.706857 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.709269 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.709743 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.709867 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.709918 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.710192 (+ 274us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.710290 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":168,"threads_started":1}
I20260805 23:15:28.710539 26362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.704763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.23.88.66:48689)
0805 23:15:28.705229 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:28.705247 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:28.705390 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:28.705808 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:28.705813 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:28.705840 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:28.706063 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.706072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.707159 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.707163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.708917 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.708928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.709914 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.709917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.710019 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.710045 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:15:28.710167 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 23:15:28.710267 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":189,"thread_start_us":163,"threads_started":1}
I20260805 23:15:28.711164 26363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.704826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:44637 (local address 127.23.88.66:59587)
0805 23:15:28.705396 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:28.705413 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:28.705555 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:28.705938 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:28.705941 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:28.705965 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:28.706095 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.706101 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.707137 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.707144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.708913 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.708928 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.710630 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.710634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.710745 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.710771 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:15:28.710873 (+ 102us) client_negotiation.cc:240] Negotiation successful
0805 23:15:28.710957 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":176,"thread_start_us":158,"threads_started":1}
I20260805 23:15:28.711447 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ae7814b83245c486339b491f4ed21a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2" is_pre_election: true
I20260805 23:15:28.711588 26366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.704961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:59587 (local address 127.23.88.67:44637)
0805 23:15:28.705238 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.705261 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.705532 (+ 271us) server_negotiation.cc:407] Connection header received
0805 23:15:28.705706 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.705709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.705794 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.705894 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:28.706255 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.706853 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.709995 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.710462 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.710637 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.710693 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.711342 (+ 649us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.711430 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":105,"threads_started":1}
I20260805 23:15:28.711679 25886 raft_consensus.cc:2454] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 0.
I20260805 23:15:28.712123 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ae7814b83245c486339b491f4ed21a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042" is_pre_election: true
I20260805 23:15:28.712214 25966 leader_election.cc:302] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d030a52b0e7146ca83143347ebc7b4d2, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:28.712288 26183 raft_consensus.cc:2454] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 0.
I20260805 23:15:28.712545 26361 raft_consensus.cc:2781] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:28.712623 26361 raft_consensus.cc:482] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:28.712675 26361 raft_consensus.cc:3037] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:28.713765 26361 raft_consensus.cc:504] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.714176 26361 leader_election.cc:288] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 election: Requested vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:28.714929 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ae7814b83245c486339b491f4ed21a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2"
I20260805 23:15:28.714962 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65ae7814b83245c486339b491f4ed21a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042"
I20260805 23:15:28.715149 26183 raft_consensus.cc:3037] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:28.715149 25886 raft_consensus.cc:3037] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:28.716233 26183 raft_consensus.cc:2454] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 1.
I20260805 23:15:28.716243 25886 raft_consensus.cc:2454] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 1.
I20260805 23:15:28.716753 25965 leader_election.cc:302] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, d030a52b0e7146ca83143347ebc7b4d2, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:28.717020 26361 raft_consensus.cc:2781] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:28.717330 26361 raft_consensus.cc:686] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 LEADER]: Becoming Leader. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Running, Role: LEADER
I20260805 23:15:28.717625 26361 consensus_queue.cc:237] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.722289 25719 catalog_manager.cc:5495] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 reported cstate change: term changed from 0 to 1, leader changed from <none> to dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "dda2f7699a194f8fbe20a4cf0298d656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:28.757491 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.752216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55778 (local address 127.23.88.66:46145)
0805 23:15:28.752547 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.752554 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.752568 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:28.752676 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.752678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.752738 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.752858 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:28.753198 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.753796 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.754895 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.755296 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.755407 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.755921 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:28.756852 (+ 931us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:28.756991 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.757192 (+ 201us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.757303 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":152,"threads_started":1}
I20260805 23:15:28.774163 26373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.769867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.23.88.66:59205)
0805 23:15:28.770242 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:28.770256 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:28.770356 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:28.770672 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:28.770674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:28.770681 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:28.770802 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.770807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.771680 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.771683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.772714 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.772725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.773266 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.773270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.773378 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.773447 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:28.773809 (+ 362us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:28.773815 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:15:28.773885 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:15:28.773989 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":149,"threads_started":1}
I20260805 23:15:28.774226 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.770152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:59205 (local address 127.23.88.65:44319)
0805 23:15:28.770300 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.770303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.770319 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:28.770505 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.770507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.770559 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.770648 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:28.770899 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.771512 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.772901 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.773129 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.773223 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.773623 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:28.773709 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:28.773773 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.773948 (+ 175us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.774032 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:28.784482 26183 raft_consensus.cc:1261] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Refusing update from remote peer dda2f7699a194f8fbe20a4cf0298d656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:28.785063 25886 raft_consensus.cc:1261] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Refusing update from remote peer dda2f7699a194f8fbe20a4cf0298d656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:28.785624 26367 consensus_queue.cc:1035] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:28.785868 26361 consensus_queue.cc:1035] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:28.810429 26378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.806338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46145 (local address 127.23.88.65:43779)
0805 23:15:28.806891 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:28.806910 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:28.807036 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:28.807357 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:28.807361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:28.807369 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:28.807499 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.807507 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.808355 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.808362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.809017 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:28.809026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.809612 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.809616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:28.809713 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.809753 (+ 40us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:28.810095 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:28.810104 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:15:28.810172 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 23:15:28.810275 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":115,"threads_started":1}
I20260805 23:15:28.810427 26370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:28.806612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:43779 (local address 127.23.88.66:46145)
0805 23:15:28.806837 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:28.806841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:28.807003 (+ 162us) server_negotiation.cc:407] Connection header received
0805 23:15:28.807193 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:28.807196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:28.807244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:28.807344 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:28.807628 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.808236 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.809161 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:28.809476 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:28.809602 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:28.809891 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:28.809987 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:28.810087 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:28.810192 (+ 105us) server_negotiation.cc:299] Negotiation successful
0805 23:15:28.810257 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260805 23:15:28.844561 25720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:28.845101 25720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:28.848016 25720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Sending DeleteTablet for 3 replicas of tablet 65ae7814b83245c486339b491f4ed21a
I20260805 23:15:28.848824 25865 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ae7814b83245c486339b491f4ed21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39102
I20260805 23:15:28.848997 26162 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ae7814b83245c486339b491f4ed21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:42596
I20260805 23:15:28.849118 26344 tablet_replica.cc:330] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: stopping tablet replica
I20260805 23:15:28.849208 26346 tablet_replica.cc:330] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: stopping tablet replica
I20260805 23:15:28.849193 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet 65ae7814b83245c486339b491f4ed21a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:28 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55774
I20260805 23:15:28.849295 26344 raft_consensus.cc:2229] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:28.849334 26346 raft_consensus.cc:2229] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:28.849408 26345 tablet_replica.cc:330] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: stopping tablet replica
I20260805 23:15:28.849443 26344 raft_consensus.cc:2258] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.849469 26346 raft_consensus.cc:2258] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.849548 26345 raft_consensus.cc:2229] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:28.849916 26345 raft_consensus.cc:2258] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:28.850345 26344 ts_tablet_manager.cc:1905] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.850317 25720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:28.850867 25720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:28.850907 26345 ts_tablet_manager.cc:1905] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.854100 26346 ts_tablet_manager.cc:1905] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:28.854336 26344 ts_tablet_manager.cc:1918] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:28.854424 26344 log.cc:1196] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/65ae7814b83245c486339b491f4ed21a
I20260805 23:15:28.854708 26344 ts_tablet_manager.cc:1939] T 65ae7814b83245c486339b491f4ed21a P d030a52b0e7146ca83143347ebc7b4d2: Deleting consensus metadata
I20260805 23:15:28.855916 25706 catalog_manager.cc:4841] TS d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319): tablet 65ae7814b83245c486339b491f4ed21a (table test-table [id=d90c8fd2107540b3aba3dd96fe006be7]) successfully deleted
I20260805 23:15:28.858084 26014 tablet_service.cc:1467] Processing CreateTablet for tablet a57e8a33de7a438b99e9c10d3762209a (DEFAULT_TABLE table=test-table [id=9f0fcd3ceb3842de8d37bc704e5c391d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.858253 26162 tablet_service.cc:1467] Processing CreateTablet for tablet a57e8a33de7a438b99e9c10d3762209a (DEFAULT_TABLE table=test-table [id=9f0fcd3ceb3842de8d37bc704e5c391d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.858317 25865 tablet_service.cc:1467] Processing CreateTablet for tablet a57e8a33de7a438b99e9c10d3762209a (DEFAULT_TABLE table=test-table [id=9f0fcd3ceb3842de8d37bc704e5c391d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:28.858433 26014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a57e8a33de7a438b99e9c10d3762209a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.858587 26162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a57e8a33de7a438b99e9c10d3762209a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.858592 25865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a57e8a33de7a438b99e9c10d3762209a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:28.862154 26357 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Bootstrap starting.
I20260805 23:15:28.863530 26357 tablet_bootstrap.cc:654] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.864017 26355 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2: Bootstrap starting.
I20260805 23:15:28.864012 26346 ts_tablet_manager.cc:1918] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:28.864333 26346 log.cc:1196] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/65ae7814b83245c486339b491f4ed21a
I20260805 23:15:28.864775 26346 ts_tablet_manager.cc:1939] T 65ae7814b83245c486339b491f4ed21a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting consensus metadata
I20260805 23:15:28.865762 26355 tablet_bootstrap.cc:654] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.866247 25705 catalog_manager.cc:4841] TS 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637): tablet 65ae7814b83245c486339b491f4ed21a (table test-table [id=d90c8fd2107540b3aba3dd96fe006be7]) successfully deleted
I20260805 23:15:28.867054 26356 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Bootstrap starting.
I20260805 23:15:28.867446 26345 ts_tablet_manager.cc:1918] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:28.867553 26345 log.cc:1196] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/65ae7814b83245c486339b491f4ed21a
I20260805 23:15:28.867981 26345 ts_tablet_manager.cc:1939] T 65ae7814b83245c486339b491f4ed21a P dda2f7699a194f8fbe20a4cf0298d656: Deleting consensus metadata
I20260805 23:15:28.868626 26356 tablet_bootstrap.cc:654] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:28.869155 25705 catalog_manager.cc:4841] TS dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145): tablet 65ae7814b83245c486339b491f4ed21a (table test-table [id=d90c8fd2107540b3aba3dd96fe006be7]) successfully deleted
I20260805 23:15:28.870240 26357 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: No bootstrap required, opened a new log
I20260805 23:15:28.870352 26356 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: No bootstrap required, opened a new log
I20260805 23:15:28.870393 26357 ts_tablet_manager.cc:1397] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260805 23:15:28.870445 26356 ts_tablet_manager.cc:1397] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260805 23:15:28.870563 26355 tablet_bootstrap.cc:492] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2: No bootstrap required, opened a new log
I20260805 23:15:28.870661 26355 ts_tablet_manager.cc:1397] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260805 23:15:28.870913 26356 raft_consensus.cc:348] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.871040 26356 raft_consensus.cc:374] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.870958 26357 raft_consensus.cc:348] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.871073 26356 raft_consensus.cc:729] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d14ca37dd614e3a8e0f6432006dc042, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.871127 26357 raft_consensus.cc:374] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.871176 26357 raft_consensus.cc:729] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.871173 26355 raft_consensus.cc:348] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.871294 26356 consensus_queue.cc:260] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.871299 26355 raft_consensus.cc:374] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:28.871694 26356 ts_tablet_manager.cc:1428] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:15:28.871560 26355 raft_consensus.cc:729] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d030a52b0e7146ca83143347ebc7b4d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:28.871887 26355 consensus_queue.cc:260] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.872335 26355 ts_tablet_manager.cc:1428] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:28.872784 26357 consensus_queue.cc:260] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:28.873179 26357 ts_tablet_manager.cc:1428] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:29.049544 26361 raft_consensus.cc:482] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:29.049731 26361 raft_consensus.cc:504] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:29.050192 26361 leader_election.cc:288] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:29.051015 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a57e8a33de7a438b99e9c10d3762209a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042" is_pre_election: true
I20260805 23:15:29.051016 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a57e8a33de7a438b99e9c10d3762209a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2" is_pre_election: true
I20260805 23:15:29.051254 25886 raft_consensus.cc:2454] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 0.
I20260805 23:15:29.051254 26183 raft_consensus.cc:2454] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 0.
I20260805 23:15:29.051707 25965 leader_election.cc:302] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:29.051944 26361 raft_consensus.cc:2781] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:29.052004 26361 raft_consensus.cc:482] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:29.052039 26361 raft_consensus.cc:3037] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:29.053095 26361 raft_consensus.cc:504] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:29.053486 26361 leader_election.cc:288] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 election: Requested vote from peers d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319), 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637)
I20260805 23:15:29.053957 26183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a57e8a33de7a438b99e9c10d3762209a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d14ca37dd614e3a8e0f6432006dc042"
I20260805 23:15:29.054097 26183 raft_consensus.cc:3037] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:29.053987 25886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a57e8a33de7a438b99e9c10d3762209a" candidate_uuid: "dda2f7699a194f8fbe20a4cf0298d656" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d030a52b0e7146ca83143347ebc7b4d2"
I20260805 23:15:29.054194 25886 raft_consensus.cc:3037] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:29.055258 26183 raft_consensus.cc:2454] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 1.
I20260805 23:15:29.055430 25886 raft_consensus.cc:2454] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dda2f7699a194f8fbe20a4cf0298d656 in term 1.
I20260805 23:15:29.055827 25965 leader_election.cc:302] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9d14ca37dd614e3a8e0f6432006dc042, d030a52b0e7146ca83143347ebc7b4d2, dda2f7699a194f8fbe20a4cf0298d656; no voters:
I20260805 23:15:29.056217 26361 raft_consensus.cc:2781] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:29.056358 26361 raft_consensus.cc:686] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 LEADER]: Becoming Leader. State: Replica: dda2f7699a194f8fbe20a4cf0298d656, State: Running, Role: LEADER
I20260805 23:15:29.056555 26361 consensus_queue.cc:237] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } }
I20260805 23:15:29.059374 25720 catalog_manager.cc:5495] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 reported cstate change: term changed from 0 to 1, leader changed from <none> to dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "dda2f7699a194f8fbe20a4cf0298d656" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dda2f7699a194f8fbe20a4cf0298d656" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:29.087589 26183 raft_consensus.cc:1261] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Refusing update from remote peer dda2f7699a194f8fbe20a4cf0298d656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:29.087812 25886 raft_consensus.cc:1261] T a57e8a33de7a438b99e9c10d3762209a P d030a52b0e7146ca83143347ebc7b4d2 [term 1 FOLLOWER]: Refusing update from remote peer dda2f7699a194f8fbe20a4cf0298d656: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:29.088335 26367 consensus_queue.cc:1035] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d14ca37dd614e3a8e0f6432006dc042" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 44637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:29.088537 26361 consensus_queue.cc:1035] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d030a52b0e7146ca83143347ebc7b4d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260805 23:15:29.107158 26328 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260805 23:15:30.683161 25719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:30.683341 25719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:30.686571 25719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 217e6ab533884f438a6e62a58cc976b1: Sending DeleteTablet for 3 replicas of tablet a57e8a33de7a438b99e9c10d3762209a
I20260805 23:15:30.687417 25865 tablet_service.cc:1514] Processing DeleteTablet for tablet a57e8a33de7a438b99e9c10d3762209a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39102
I20260805 23:15:30.687516 26014 tablet_service.cc:1514] Processing DeleteTablet for tablet a57e8a33de7a438b99e9c10d3762209a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55774
I20260805 23:15:30.687785 26162 tablet_service.cc:1514] Processing DeleteTablet for tablet a57e8a33de7a438b99e9c10d3762209a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:42596
I20260805 23:15:30.687880 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25798
I20260805 23:15:30.687925 26410 tablet_replica.cc:330] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: stopping tablet replica
I20260805 23:15:30.689806 26410 raft_consensus.cc:2229] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:30.690179 26410 raft_consensus.cc:2258] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:30.691208 26410 ts_tablet_manager.cc:1905] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:30.691438 26411 tablet_replica.cc:330] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: stopping tablet replica
I20260805 23:15:30.691677 26411 raft_consensus.cc:2229] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:30.691859 26411 raft_consensus.cc:2258] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:30.693137 26411 ts_tablet_manager.cc:1905] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:30.695204 26410 ts_tablet_manager.cc:1918] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:15:30.695307 26410 log.cc:1196] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/a57e8a33de7a438b99e9c10d3762209a
I20260805 23:15:30.695552 26411 ts_tablet_manager.cc:1918] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260805 23:15:30.695672 26411 log.cc:1196] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/a57e8a33de7a438b99e9c10d3762209a
I20260805 23:15:30.695691 26410 ts_tablet_manager.cc:1939] T a57e8a33de7a438b99e9c10d3762209a P dda2f7699a194f8fbe20a4cf0298d656: Deleting consensus metadata
I20260805 23:15:30.695956 26411 ts_tablet_manager.cc:1939] T a57e8a33de7a438b99e9c10d3762209a P 9d14ca37dd614e3a8e0f6432006dc042: Deleting consensus metadata
I20260805 23:15:30.696669 25705 catalog_manager.cc:4841] TS dda2f7699a194f8fbe20a4cf0298d656 (127.23.88.66:46145): tablet a57e8a33de7a438b99e9c10d3762209a (table test-table [id=9f0fcd3ceb3842de8d37bc704e5c391d]) successfully deleted
I20260805 23:15:30.696902 25705 catalog_manager.cc:4841] TS 9d14ca37dd614e3a8e0f6432006dc042 (127.23.88.67:44637): tablet a57e8a33de7a438b99e9c10d3762209a (table test-table [id=9f0fcd3ceb3842de8d37bc704e5c391d]) successfully deleted
W20260805 23:15:30.699719 25706 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:44319 (error 108)
I20260805 23:15:30.700042 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25946
I20260805 23:15:30.702232 26413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:30.701606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44319 (local address 127.0.0.1:32826)
0805 23:15:30.701959 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:30.702064 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:44319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":108,"threads_started":1}
W20260805 23:15:30.702571 25706 catalog_manager.cc:4584] TS d030a52b0e7146ca83143347ebc7b4d2 (127.23.88.65:44319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a57e8a33de7a438b99e9c10d3762209a: Network error: Client connection negotiation failed: client connection to 127.23.88.65:44319: connect: Connection refused (error 111)
I20260805 23:15:30.709370 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26094
I20260805 23:15:30.718078 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 25688
2026-08-05T23:15:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6875 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:30 dist-test-slave-vkjp krb5kdc[26416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:30 dist-test-slave-vkjp krb5kdc[26416](info): set up 2 sockets
Aug 05 23:15:30 dist-test-slave-vkjp krb5kdc[26416](info): commencing operation
krb5kdc: starting...
W20260805 23:15:31.809476 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:31 dist-test-slave-vkjp krb5kdc[26416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:31.947614 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34147
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:32.072218 26430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:32.072613 26430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:32.072692 26430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:32.077081 26430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:32.077206 26430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:32.077250 26430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:32.077287 26430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:32.077325 26430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:32.083314 26430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:32.084631 26430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:32.085773 26430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:32.096874 26430 server_base.cc:1034] running on GCE node
W20260805 23:15:32.096800 26436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.096812 26435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.096848 26438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.097849 26430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:32.099200 26430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:32.100423 26430 hybrid_clock.cc:648] HybridClock initialized: now 1785971732100374 us; error 68 us; skew 500 ppm
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:32.108700 26430 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:32.109789 26430 webserver.cc:460] Webserver started at http://127.23.88.126:32801/ using document root <none> and password file <none>
I20260805 23:15:32.110380 26430 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:32.110448 26430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:32.110633 26430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:32.112979 26430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "7981a3f4ec0841a9a7335cdb5122cbf8"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "4e930d3255ab8685c941c5e6cda503b0"
server_key_iv: "93c50db999fe0f7b65d67808471ba9ca"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.113574 26430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "7981a3f4ec0841a9a7335cdb5122cbf8"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "4e930d3255ab8685c941c5e6cda503b0"
server_key_iv: "93c50db999fe0f7b65d67808471ba9ca"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.118388 26430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260805 23:15:32.121338 26445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.122843 26430 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:32.122987 26430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "7981a3f4ec0841a9a7335cdb5122cbf8"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "4e930d3255ab8685c941c5e6cda503b0"
server_key_iv: "93c50db999fe0f7b65d67808471ba9ca"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.123199 26430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:32.147292 26430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:32.150941 26430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:32.151224 26430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:32.159418 26430 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34147
I20260805 23:15:32.159444 26498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34147 every 8 connection(s)
I20260805 23:15:32.160526 26430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:32.163676 26499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:32.165961 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26430
I20260805 23:15:32.166095 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:32.166369 23905 external_mini_cluster.cc:1428] Setting key 64b927187f81acafe36befcce78f299a
I20260805 23:15:32.171532 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8: Bootstrap starting.
I20260805 23:15:32.174536 26499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:32.175709 26499 log.cc:824] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:32.178737 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8: No bootstrap required, opened a new log
I20260805 23:15:32.183643 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.168028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.23.88.126:34147)
0805 23:15:32.168487 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:32.168504 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:32.168541 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:15:32.169275 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:32.169295 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:32.169662 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:32.169964 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:32.171035 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.172216 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.173457 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.173979 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.174112 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.177598 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:32.177605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:32.177624 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:32.177658 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:32.179837 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.180415 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.180421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.180428 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.180588 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.180928 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.180938 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.180940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.181611 (+ 671us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:32.181826 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:32.182144 (+ 318us) server_negotiation.cc:299] Negotiation successful
0805 23:15:32.182479 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":161,"threads_started":1}
I20260805 23:15:32.183842 26499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } }
I20260805 23:15:32.184159 26499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:32.184224 26499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7981a3f4ec0841a9a7335cdb5122cbf8, State: Initialized, Role: FOLLOWER
I20260805 23:15:32.184733 26499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } }
I20260805 23:15:32.184903 26499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:32.184975 26499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:32.185079 26499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:32.186133 26499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } }
I20260805 23:15:32.186533 26499 leader_election.cc:302] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7981a3f4ec0841a9a7335cdb5122cbf8; no voters:
I20260805 23:15:32.186889 26499 leader_election.cc:288] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:32.187031 26504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:32.187336 26504 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 7981a3f4ec0841a9a7335cdb5122cbf8, State: Running, Role: LEADER
I20260805 23:15:32.187680 26504 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } }
I20260805 23:15:32.188289 26499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:32.189528 26506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7981a3f4ec0841a9a7335cdb5122cbf8. Latest consensus state: current_term: 1 leader_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } } }
I20260805 23:15:32.189666 26506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:32.189962 26505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7981a3f4ec0841a9a7335cdb5122cbf8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34147 } } }
I20260805 23:15:32.190054 26505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:32.190078 26513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:32.193543 26513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:32.199566 26513 catalog_manager.cc:1347] Generated new cluster ID: a941c572c51a489db09926892153646a
I20260805 23:15:32.199671 26513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:32.225679 26513 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:32.227144 26513 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:32.234560 26513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7981a3f4ec0841a9a7335cdb5122cbf8: Generated new TSK 0
I20260805 23:15:32.235527 26513 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:32.271745 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:32.393179 26526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:32.393513 26526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:32.393560 26526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:32.397966 26526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:32.398065 26526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:32.398129 26526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:32.404271 26526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:32.405485 26526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:32.406603 26526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:32.417718 26532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.417793 26526 server_base.cc:1034] running on GCE node
W20260805 23:15:32.417747 26531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.417718 26534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.418471 26526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:32.419340 26526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:32.420564 26526 hybrid_clock.cc:648] HybridClock initialized: now 1785971732420541 us; error 36 us; skew 500 ppm
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:32.424039 26526 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:32.425096 26526 webserver.cc:460] Webserver started at http://127.23.88.65:45123/ using document root <none> and password file <none>
I20260805 23:15:32.425735 26526 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:32.425804 26526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:32.426040 26526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:32.428387 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "28da42b5e7f64c6e93df845d58be2a29"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "da837a062878fd664ef1cd046c03d344"
server_key_iv: "45197148808ab73d15690875e913f9c2"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.429018 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "28da42b5e7f64c6e93df845d58be2a29"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "da837a062878fd664ef1cd046c03d344"
server_key_iv: "45197148808ab73d15690875e913f9c2"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.433171 26526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:32.436040 26542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.437574 26526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:15:32.437755 26526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "28da42b5e7f64c6e93df845d58be2a29"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "da837a062878fd664ef1cd046c03d344"
server_key_iv: "45197148808ab73d15690875e913f9c2"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.437947 26526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:32.460438 26526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:32.464046 26526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:32.464278 26526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:32.464979 26526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:32.465901 26526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:32.465957 26526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.466033 26526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:32.466075 26526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.475898 26526 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:34299
I20260805 23:15:32.475917 26660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:34299 every 8 connection(s)
I20260805 23:15:32.476914 26526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:32.480305 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26526
I20260805 23:15:32.480441 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:32.480758 23905 external_mini_cluster.cc:1428] Setting key f0a9502c0252d74c64dbe72e4629f96e
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:32.495036 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.479006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40363 (local address 127.23.88.126:34147)
0805 23:15:32.479229 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:32.479232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:32.480303 (+ 1071us) server_negotiation.cc:407] Connection header received
0805 23:15:32.482243 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:32.482246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:32.482300 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:32.482399 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:32.484228 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.484826 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.486536 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.487002 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.487179 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.490850 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:32.490853 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:32.490856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:32.490884 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:32.492522 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.493145 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.493149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.493151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.493239 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.493625 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.493629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.493631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.493962 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:32.494091 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:32.494678 (+ 587us) server_negotiation.cc:299] Negotiation successful
0805 23:15:32.494845 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:32.495975 26663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.479441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34147 (local address 127.23.88.65:40363)
0805 23:15:32.480063 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:32.480137 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:32.482000 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:32.482622 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:32.482633 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:32.483246 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:32.484038 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.484063 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.484980 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.484985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.486372 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.486384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.487254 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.487258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.487457 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.488267 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:32.488305 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:32.490601 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:32.492677 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.492685 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.492704 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.493023 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.493348 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.493351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.493353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.493518 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.494089 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:32.494097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:32.494397 (+ 300us) client_negotiation.cc:769] Sending connection context
0805 23:15:32.494622 (+ 225us) client_negotiation.cc:240] Negotiation successful
0805 23:15:32.495008 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":128,"threads_started":1}
I20260805 23:15:32.497428 26661 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34147
I20260805 23:15:32.497735 26661 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:32.498306 26661 heartbeater.cc:511] Master 127.23.88.126:34147 requested a full tablet report, sending...
I20260805 23:15:32.499974 26463 ts_manager.cc:194] Registered new tserver with Master: 28da42b5e7f64c6e93df845d58be2a29 (127.23.88.65:34299)
I20260805 23:15:32.501202 26463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:40363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:32.514770 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:32.634620 26666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:32.634927 26666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:32.634974 26666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:32.639459 26666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:32.639542 26666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:32.639642 26666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:32.645524 26666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:32.646727 26666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:32.647838 26666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:32.658794 26674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.658794 26671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.659237 26666 server_base.cc:1034] running on GCE node
W20260805 23:15:32.658813 26672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.659858 26666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:32.660698 26666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:32.661906 26666 hybrid_clock.cc:648] HybridClock initialized: now 1785971732661878 us; error 48 us; skew 500 ppm
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:32.665228 26666 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:32.666246 26666 webserver.cc:460] Webserver started at http://127.23.88.66:35571/ using document root <none> and password file <none>
I20260805 23:15:32.666831 26666 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:32.666904 26666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:32.667143 26666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:32.669390 26666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "bdadeb5eb40d48e0aba3631a83b7a7a4"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "b71ad6ed36b4266224a1f86717feb412"
server_key_iv: "51638182f577cbd82518c8827992dc14"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.669958 26666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "bdadeb5eb40d48e0aba3631a83b7a7a4"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "b71ad6ed36b4266224a1f86717feb412"
server_key_iv: "51638182f577cbd82518c8827992dc14"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.674216 26666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:15:32.677188 26681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.678627 26666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:32.678813 26666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "bdadeb5eb40d48e0aba3631a83b7a7a4"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "b71ad6ed36b4266224a1f86717feb412"
server_key_iv: "51638182f577cbd82518c8827992dc14"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.678949 26666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:32.715281 26666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:32.718672 26666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:32.718897 26666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:32.719614 26666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:32.720520 26666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:32.720570 26666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.720641 26666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:32.720675 26666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.731575 26666 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:38053
I20260805 23:15:32.731621 26799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:38053 every 8 connection(s)
I20260805 23:15:32.732575 26666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:32.742940 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26666
I20260805 23:15:32.743120 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:32.743474 23905 external_mini_cluster.cc:1428] Setting key 9d30fcc71c9e0c480e8bd24d3dd49e38
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:32.749024 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.734927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:59677 (local address 127.23.88.126:34147)
0805 23:15:32.735148 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:32.735153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:32.736137 (+ 984us) server_negotiation.cc:407] Connection header received
0805 23:15:32.737599 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:32.737602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:32.737660 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:32.737747 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:32.739211 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.739759 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.740999 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.741231 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.741356 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.744488 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:32.744491 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:32.744493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:32.744518 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:32.746318 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.747037 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.747041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.747043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.747187 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.747672 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.747676 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.747679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.747933 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:32.748071 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:32.748666 (+ 595us) server_negotiation.cc:299] Negotiation successful
0805 23:15:32.748823 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:15:32.750054 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.735443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34147 (local address 127.23.88.66:59677)
0805 23:15:32.735944 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:32.736004 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:32.737362 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:32.737964 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:32.737971 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:32.738369 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:32.739016 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.739032 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.739916 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.739920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.740837 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.740847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.741410 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.741414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.741660 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.742404 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:32.742422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:32.744308 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:32.746492 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.746499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.746515 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.746873 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.747347 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.747352 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.747354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.747538 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.748059 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:32.748067 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:32.748378 (+ 311us) client_negotiation.cc:769] Sending connection context
0805 23:15:32.748601 (+ 223us) client_negotiation.cc:240] Negotiation successful
0805 23:15:32.748994 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":119,"threads_started":1}
I20260805 23:15:32.751641 26800 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34147
I20260805 23:15:32.751924 26800 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:32.752600 26800 heartbeater.cc:511] Master 127.23.88.126:34147 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:32.753937 26463 ts_manager.cc:194] Registered new tserver with Master: bdadeb5eb40d48e0aba3631a83b7a7a4 (127.23.88.66:38053)
I20260805 23:15:32.754626 26463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:59677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:32.776063 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:32.897482 26805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:32.897799 26805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:32.897845 26805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:32.902117 26805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:32.902204 26805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:32.902268 26805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:32.908067 26805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:32.909317 26805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:32.910451 26805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:32.920580 26815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.920637 26812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:32.920625 26811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:32.920773 26805 server_base.cc:1034] running on GCE node
I20260805 23:15:32.921355 26805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:32.922081 26805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:32.923271 26805 hybrid_clock.cc:648] HybridClock initialized: now 1785971732923256 us; error 26 us; skew 500 ppm
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:32.926149 26805 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:32.927160 26805 webserver.cc:460] Webserver started at http://127.23.88.67:39383/ using document root <none> and password file <none>
I20260805 23:15:32.927790 26805 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:32.927853 26805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:32.928027 26805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:32.930202 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "6147195df0c145c38f23d4d7b5a99276"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "6c534df5adabf94abaec7f7fe241e783"
server_key_iv: "270d7b2fa676f6fc11c15c6933edde1b"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.930764 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "6147195df0c145c38f23d4d7b5a99276"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "6c534df5adabf94abaec7f7fe241e783"
server_key_iv: "270d7b2fa676f6fc11c15c6933edde1b"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.934796 26805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:15:32.937662 26822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.939244 26805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260805 23:15:32.939422 26805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "6147195df0c145c38f23d4d7b5a99276"
format_stamp: "Formatted at 2026-08-05 23:15:32 on dist-test-slave-vkjp"
server_key: "6c534df5adabf94abaec7f7fe241e783"
server_key_iv: "270d7b2fa676f6fc11c15c6933edde1b"
server_key_version: "encryptionkey@0"
I20260805 23:15:32.939561 26805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:32.960525 26805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:32.964035 26805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:32.964279 26805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:32.964927 26805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:32.965811 26805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:32.965885 26805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.965986 26805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:32.966035 26805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:32.975803 26805 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:45515
I20260805 23:15:32.975826 26943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:45515 every 8 connection(s)
I20260805 23:15:32.976801 26805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:32.984210 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26805
I20260805 23:15:32.984349 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:32.984658 23905 external_mini_cluster.cc:1428] Setting key 467967df8781d36090c65555c86bcda9
Aug 05 23:15:32 dist-test-slave-vkjp krb5kdc[26416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971732, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:32.992847 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.978937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:52923 (local address 127.23.88.126:34147)
0805 23:15:32.979139 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:32.979145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:32.980118 (+ 973us) server_negotiation.cc:407] Connection header received
0805 23:15:32.981526 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:32.981530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:32.981596 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:32.981704 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:32.983051 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.983714 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.985356 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.985607 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.985741 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.988824 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:32.988827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:32.988830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:32.988858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:32.990634 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.991344 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.991347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.991348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.991433 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:32.991776 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:32.991778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:32.991779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:32.991997 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:32.992076 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:32.992601 (+ 525us) server_negotiation.cc:299] Negotiation successful
0805 23:15:32.992716 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:32.993769 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:32.979427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34147 (local address 127.23.88.67:52923)
0805 23:15:32.979939 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:32.979993 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:32.981277 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:32.981894 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:32.981903 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:32.982291 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:32.982882 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.982917 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.983841 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.983844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.985190 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:32.985202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:32.985783 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:32.985787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:32.985989 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:32.986746 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:32.986763 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:32.988643 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:32.990790 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.990796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.990811 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.991237 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.991524 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:32.991526 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:32.991527 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:32.991673 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:32.992088 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:32.992093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:32.992373 (+ 280us) client_negotiation.cc:769] Sending connection context
0805 23:15:32.992536 (+ 163us) client_negotiation.cc:240] Negotiation successful
0805 23:15:32.992820 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":131,"threads_started":1}
I20260805 23:15:32.995019 26944 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34147
I20260805 23:15:32.995290 26944 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:32.995807 26944 heartbeater.cc:511] Master 127.23.88.126:34147 requested a full tablet report, sending...
I20260805 23:15:32.996944 26463 ts_manager.cc:194] Registered new tserver with Master: 6147195df0c145c38f23d4d7b5a99276 (127.23.88.67:45515)
I20260805 23:15:32.997696 26463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:52923
I20260805 23:15:32.999171 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:15:33.015981 26502 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:33.013932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.23.88.126:34147)
0805 23:15:33.014168 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:33.014174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:33.014501 (+ 327us) server_negotiation.cc:407] Connection header received
0805 23:15:33.014682 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:33.014686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:33.014744 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:33.014849 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:33.015749 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42996: BlockingRecv error: recv got EOF from 127.0.0.1:42996 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
I20260805 23:15:33.017428 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26526
I20260805 23:15:33.028375 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26666
I20260805 23:15:33.037410 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26805
I20260805 23:15:33.044759 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26430
2026-08-05T23:15:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2325 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:33 dist-test-slave-vkjp krb5kdc[26957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:33 dist-test-slave-vkjp krb5kdc[26957](info): set up 2 sockets
Aug 05 23:15:33 dist-test-slave-vkjp krb5kdc[26957](info): commencing operation
krb5kdc: starting...
W20260805 23:15:34.141450 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:34.275314 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42851
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:34.392419 26971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:34.392795 26971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:34.392887 26971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:34.397194 26971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:34.397322 26971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:34.397364 26971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:34.397411 26971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:34.397449 26971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:34.403931 26971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:34.405194 26971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:34.406335 26971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:34.415868 26976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:34.415912 26979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:34.415966 26971 server_base.cc:1034] running on GCE node
W20260805 23:15:34.415884 26977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:34.416864 26971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:34.418144 26971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:34.419322 26971 hybrid_clock.cc:648] HybridClock initialized: now 1785971734419296 us; error 46 us; skew 500 ppm
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[26957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971734, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:34.422634 26971 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:34.423837 26971 webserver.cc:460] Webserver started at http://127.23.88.126:36071/ using document root <none> and password file <none>
I20260805 23:15:34.424476 26971 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:34.424567 26971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:34.424809 26971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:34.427160 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "06dad875435f482ba527fe44d4868650"
format_stamp: "Formatted at 2026-08-05 23:15:34 on dist-test-slave-vkjp"
server_key: "690e4c0d02b3f2e5e0fefe001384ea79"
server_key_iv: "a203181ccc8990b3e6b0d7b80b995174"
server_key_version: "encryptionkey@0"
I20260805 23:15:34.427829 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "06dad875435f482ba527fe44d4868650"
format_stamp: "Formatted at 2026-08-05 23:15:34 on dist-test-slave-vkjp"
server_key: "690e4c0d02b3f2e5e0fefe001384ea79"
server_key_iv: "a203181ccc8990b3e6b0d7b80b995174"
server_key_version: "encryptionkey@0"
I20260805 23:15:34.432004 26971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:34.434748 26986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:34.436199 26971 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:34.436400 26971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "06dad875435f482ba527fe44d4868650"
format_stamp: "Formatted at 2026-08-05 23:15:34 on dist-test-slave-vkjp"
server_key: "690e4c0d02b3f2e5e0fefe001384ea79"
server_key_iv: "a203181ccc8990b3e6b0d7b80b995174"
server_key_version: "encryptionkey@0"
I20260805 23:15:34.436548 26971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:34.457461 26971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:34.460913 26971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:34.461189 26971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:34.468950 26971 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42851
I20260805 23:15:34.469352 27043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42851 every 8 connection(s)
I20260805 23:15:34.470042 26971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:34.472924 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 26971
I20260805 23:15:34.473089 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:34.473125 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:34.473438 23905 external_mini_cluster.cc:1428] Setting key 432466272899d8cfcad4d42a39aec053
I20260805 23:15:34.481279 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650: Bootstrap starting.
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[26957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:34.484277 27044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:34.485216 27044 log.cc:824] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:34.487689 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650: No bootstrap required, opened a new log
I20260805 23:15:34.490479 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:34.474958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46568 (local address 127.23.88.126:42851)
0805 23:15:34.475564 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:34.475583 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:34.475630 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:15:34.476430 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:34.476457 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:34.476930 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:34.477345 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:34.478095 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:34.479635 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:34.480727 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:34.481473 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:34.481631 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:34.484580 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:34.484586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:34.484601 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:34.484637 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:34.487242 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:34.487832 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:34.487838 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:34.487845 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:34.488005 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:34.488323 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:34.488327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:34.488329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:34.488808 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:34.489048 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:34.489350 (+ 302us) server_negotiation.cc:299] Negotiation successful
0805 23:15:34.489692 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":172,"threads_started":1}
I20260805 23:15:34.491707 27044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } }
I20260805 23:15:34.491982 27044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:34.492066 27044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06dad875435f482ba527fe44d4868650, State: Initialized, Role: FOLLOWER
I20260805 23:15:34.492594 27044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } }
I20260805 23:15:34.492767 27044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:34.492854 27044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:34.492959 27044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:34.493638 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:34.494091 27044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } }
I20260805 23:15:34.494468 27044 leader_election.cc:302] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06dad875435f482ba527fe44d4868650; no voters:
I20260805 23:15:34.494836 27044 leader_election.cc:288] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:34.494938 27049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:34.495162 27049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [term 1 LEADER]: Becoming Leader. State: Replica: 06dad875435f482ba527fe44d4868650, State: Running, Role: LEADER
I20260805 23:15:34.495543 27049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } }
I20260805 23:15:34.496551 27044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:34.498051 27050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06dad875435f482ba527fe44d4868650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } } }
I20260805 23:15:34.498204 27050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:34.498612 27051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06dad875435f482ba527fe44d4868650. Latest consensus state: current_term: 1 leader_uuid: "06dad875435f482ba527fe44d4868650" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06dad875435f482ba527fe44d4868650" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42851 } } }
I20260805 23:15:34.498682 27059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:34.498708 27051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:34.502585 27059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:34.508886 27059 catalog_manager.cc:1347] Generated new cluster ID: 275783bb587443e29e93a172c572277a
I20260805 23:15:34.509060 27059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:34.519301 27059 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:34.520519 27059 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:34.521111 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:34.514905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46570 (local address 127.23.88.126:42851)
0805 23:15:34.515046 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:34.515051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:34.515330 (+ 279us) server_negotiation.cc:407] Connection header received
0805 23:15:34.515464 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:34.515468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:34.515524 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:34.515629 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:34.516306 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:34.517158 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:34.518395 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:34.518772 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:34.518883 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:34.519935 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:34.519939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:34.519942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:34.520519 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:34.520556 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:34.520578 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:34.520671 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:34.520851 (+ 180us) server_negotiation.cc:299] Negotiation successful
0805 23:15:34.520942 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":59}
I20260805 23:15:34.526221 27059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06dad875435f482ba527fe44d4868650: Generated new TSK 0
I20260805 23:15:34.526976 27059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:34.533704 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 26971
2026-08-05T23:15:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1486 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[27078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[27078](info): set up 2 sockets
Aug 05 23:15:34 dist-test-slave-vkjp krb5kdc[27078](info): commencing operation
krb5kdc: starting...
W20260805 23:15:35.661511 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:35 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:35.794557 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33421
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:33421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:35.914325 27091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:35.914659 27091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:35.914707 27091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:35.918991 27091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:35.919118 27091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:35.919143 27091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:35.919164 27091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:35.919183 27091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:35.924959 27091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:33421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:35.926142 27091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:35.927258 27091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:35.936493 27096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:35.936515 27099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:35.936563 27097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:35.936815 27091 server_base.cc:1034] running on GCE node
I20260805 23:15:35.937516 27091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:35.938642 27091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:35.939824 27091 hybrid_clock.cc:648] HybridClock initialized: now 1785971735939805 us; error 31 us; skew 500 ppm
Aug 05 23:15:35 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971735, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:35.942862 27091 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:35.943955 27091 webserver.cc:460] Webserver started at http://127.23.88.126:38813/ using document root <none> and password file <none>
I20260805 23:15:35.944554 27091 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:35.944621 27091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:35.944875 27091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:35.947314 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "094b81a09bd548439762ece2977bd56d"
format_stamp: "Formatted at 2026-08-05 23:15:35 on dist-test-slave-vkjp"
server_key: "2d10b6f5546d2fb54bb04448787e30d0"
server_key_iv: "22939b773883cdbd59c70a2b817b4e71"
server_key_version: "encryptionkey@0"
I20260805 23:15:35.947947 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "094b81a09bd548439762ece2977bd56d"
format_stamp: "Formatted at 2026-08-05 23:15:35 on dist-test-slave-vkjp"
server_key: "2d10b6f5546d2fb54bb04448787e30d0"
server_key_iv: "22939b773883cdbd59c70a2b817b4e71"
server_key_version: "encryptionkey@0"
I20260805 23:15:35.952073 27091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:15:35.954667 27106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:35.956142 27091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:15:35.956315 27091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "094b81a09bd548439762ece2977bd56d"
format_stamp: "Formatted at 2026-08-05 23:15:35 on dist-test-slave-vkjp"
server_key: "2d10b6f5546d2fb54bb04448787e30d0"
server_key_iv: "22939b773883cdbd59c70a2b817b4e71"
server_key_version: "encryptionkey@0"
I20260805 23:15:35.956454 27091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:35.980010 27091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:35.983451 27091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:35.983683 27091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:35.991832 27091 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:33421
I20260805 23:15:35.992120 27159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:33421 every 8 connection(s)
I20260805 23:15:35.992966 27091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:35.996244 27160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:36.002286 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27091
I20260805 23:15:36.002421 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:36.002703 23905 external_mini_cluster.cc:1428] Setting key 073a9cdf7e47059f619a6e6252541afa
I20260805 23:15:36.002873 27160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d: Bootstrap starting.
I20260805 23:15:36.005755 27160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:36.006844 27160 log.cc:824] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:36.009406 27160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d: No bootstrap required, opened a new log
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:36.014717 27160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } }
I20260805 23:15:36.015159 27160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:36.015270 27160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094b81a09bd548439762ece2977bd56d, State: Initialized, Role: FOLLOWER
I20260805 23:15:36.015852 27160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } }
I20260805 23:15:36.016041 27160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:36.016125 27160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:36.016238 27160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:36.017318 27160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } }
I20260805 23:15:36.017735 27160 leader_election.cc:302] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 094b81a09bd548439762ece2977bd56d; no voters:
I20260805 23:15:36.018182 27160 leader_election.cc:288] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:36.018312 27166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:36.018553 27166 raft_consensus.cc:686] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [term 1 LEADER]: Becoming Leader. State: Replica: 094b81a09bd548439762ece2977bd56d, State: Running, Role: LEADER
I20260805 23:15:36.018973 27166 consensus_queue.cc:237] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } }
I20260805 23:15:36.019336 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.004277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39142 (local address 127.23.88.126:33421)
0805 23:15:36.004754 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:36.004770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:36.004808 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:15:36.005503 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:36.005521 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:36.005864 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:36.006221 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:36.007465 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.008633 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.009902 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.010410 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.010574 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.013816 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:36.013821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:36.013837 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:36.013871 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:36.015938 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.016437 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.016443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.016449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.016576 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.016880 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.016884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.016886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.017315 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:36.017547 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:36.017831 (+ 284us) server_negotiation.cc:299] Negotiation successful
0805 23:15:36.018145 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":185,"threads_started":1}
I20260805 23:15:36.019807 27160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:36.020872 27167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "094b81a09bd548439762ece2977bd56d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } } }
I20260805 23:15:36.022692 27167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:36.021080 27168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 094b81a09bd548439762ece2977bd56d. Latest consensus state: current_term: 1 leader_uuid: "094b81a09bd548439762ece2977bd56d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "094b81a09bd548439762ece2977bd56d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33421 } } }
I20260805 23:15:36.023120 27168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:36.023244 27171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:36.026530 27171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:36.033926 27171 catalog_manager.cc:1347] Generated new cluster ID: 2ddde3da4ba94ed8b13ad073164a7721
I20260805 23:15:36.034031 27171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:36.067842 27171 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:36.069145 27171 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:36.076797 27171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 094b81a09bd548439762ece2977bd56d: Generated new TSK 0
I20260805 23:15:36.077615 27171 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:36.111372 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:36.230859 27189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:36.231230 27189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:36.231333 27189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:36.235841 27189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:36.235935 27189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:36.236035 27189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:36.241583 27189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:36.242691 27189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:36.243734 27189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:36.254070 27195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.254305 27189 server_base.cc:1034] running on GCE node
W20260805 23:15:36.254114 27197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:36.254070 27194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.254877 27189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:36.255698 27189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:36.256893 27189 hybrid_clock.cc:648] HybridClock initialized: now 1785971736256869 us; error 35 us; skew 500 ppm
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:36.259967 27189 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:36.261121 27189 webserver.cc:460] Webserver started at http://127.23.88.65:42047/ using document root <none> and password file <none>
I20260805 23:15:36.261718 27189 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:36.261781 27189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:36.261953 27189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:36.264117 27189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "e91df26ea9584af1a8991a482c97dd7a"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "2917091ef59bc79726ed690d26603675"
server_key_iv: "592b5246486ddd5eb8ef0731c6eaa51a"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.264658 27189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "e91df26ea9584af1a8991a482c97dd7a"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "2917091ef59bc79726ed690d26603675"
server_key_iv: "592b5246486ddd5eb8ef0731c6eaa51a"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.268637 27189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:36.271275 27204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.272635 27189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:36.272771 27189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "e91df26ea9584af1a8991a482c97dd7a"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "2917091ef59bc79726ed690d26603675"
server_key_iv: "592b5246486ddd5eb8ef0731c6eaa51a"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.272893 27189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:36.295924 27189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:36.299453 27189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:36.299676 27189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:36.300328 27189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:36.301222 27189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:36.301299 27189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.301370 27189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:36.301450 27189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.311558 27189 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:36725
I20260805 23:15:36.311597 27324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:36725 every 8 connection(s)
I20260805 23:15:36.312503 27189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:36.319779 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27189
I20260805 23:15:36.319908 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:36.320245 23905 external_mini_cluster.cc:1428] Setting key 033d2334dfb1edbd0cc743270c4a1c5f
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:36.327762 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.314461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:42837 (local address 127.23.88.126:33421)
0805 23:15:36.314618 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:36.314622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:36.315523 (+ 901us) server_negotiation.cc:407] Connection header received
0805 23:15:36.316969 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:36.316973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:36.317036 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:36.317150 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:36.318374 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.318923 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.320261 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.320558 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.320676 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.323772 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:36.323775 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:36.323778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:36.323803 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:36.325379 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.325988 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.325992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.325994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.326084 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.326477 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.326481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.326482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.326680 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:36.326804 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:36.327436 (+ 632us) server_negotiation.cc:299] Negotiation successful
0805 23:15:36.327589 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:15:36.328662 27327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.314836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33421 (local address 127.23.88.65:42837)
0805 23:15:36.315359 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:36.315409 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:36.316734 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:36.317330 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:36.317337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:36.317719 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:36.318212 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.318226 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.319063 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.319067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.320135 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.320145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.320739 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.320744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.320968 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.321776 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:36.321802 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:36.323564 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:36.325528 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.325534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.325552 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.325859 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.326186 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.326190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.326192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.326362 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.326796 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:36.326803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:36.327130 (+ 327us) client_negotiation.cc:769] Sending connection context
0805 23:15:36.327381 (+ 251us) client_negotiation.cc:240] Negotiation successful
0805 23:15:36.327754 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":205,"threads_started":1}
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:36.330070 27325 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33421
I20260805 23:15:36.330353 27325 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:36.330862 27325 heartbeater.cc:511] Master 127.23.88.126:33421 requested a full tablet report, sending...
I20260805 23:15:36.332597 27120 ts_manager.cc:194] Registered new tserver with Master: e91df26ea9584af1a8991a482c97dd7a (127.23.88.65:36725)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:36.333969 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:42837
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:36.350723 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:36.474678 27330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:36.474984 27330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:36.475069 27330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:36.480087 27330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:36.480170 27330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:36.480284 27330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:36.486498 27330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:36.487761 27330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:36.488834 27330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:36.499688 27336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:36.499766 27338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.499888 27330 server_base.cc:1034] running on GCE node
W20260805 23:15:36.499712 27335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.500561 27330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:36.501398 27330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:36.502578 27330 hybrid_clock.cc:648] HybridClock initialized: now 1785971736502555 us; error 33 us; skew 500 ppm
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:36.505669 27330 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:36.506613 27330 webserver.cc:460] Webserver started at http://127.23.88.66:44799/ using document root <none> and password file <none>
I20260805 23:15:36.507287 27330 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:36.507359 27330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:36.507555 27330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:36.509742 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "58ed0d69df9f4ed08c6fffa258f987f1"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "005635a7d914fa7926c59d80862c7179"
server_key_iv: "0b837aa408a6e890fcb15a3a588c3f5c"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.510316 27330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "58ed0d69df9f4ed08c6fffa258f987f1"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "005635a7d914fa7926c59d80862c7179"
server_key_iv: "0b837aa408a6e890fcb15a3a588c3f5c"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.514638 27330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260805 23:15:36.517448 27346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.518923 27330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:36.519145 27330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "58ed0d69df9f4ed08c6fffa258f987f1"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "005635a7d914fa7926c59d80862c7179"
server_key_iv: "0b837aa408a6e890fcb15a3a588c3f5c"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.519276 27330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:36.534834 27330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:36.537892 27330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:36.538058 27330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:36.538625 27330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:36.539556 27330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:36.539605 27330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.539646 27330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:36.539669 27330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.549127 27330 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:43457
I20260805 23:15:36.549149 27463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:43457 every 8 connection(s)
I20260805 23:15:36.550143 27330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:36.558121 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27330
I20260805 23:15:36.558260 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:36.558565 23905 external_mini_cluster.cc:1428] Setting key 2a7c1f8df33ed0530cefb7aaac065b53
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:36.565565 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.552166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:58999 (local address 127.23.88.126:33421)
0805 23:15:36.552317 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:36.552324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:36.553208 (+ 884us) server_negotiation.cc:407] Connection header received
0805 23:15:36.554632 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:36.554635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:36.554688 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:36.554769 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:36.556126 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.556663 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.557787 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.558019 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.558119 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.561267 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:36.561273 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:36.561280 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:36.561323 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:36.562944 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.563670 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.563674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.563675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.563769 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.564199 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.564202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.564203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.564398 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:36.564476 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:36.565287 (+ 811us) server_negotiation.cc:299] Negotiation successful
0805 23:15:36.565430 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:36.566824 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.552557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33421 (local address 127.23.88.66:58999)
0805 23:15:36.553030 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:36.553077 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:36.554422 (+ 1345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:36.554961 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:36.554967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:36.555470 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:36.555979 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.555993 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.556776 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.556779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.557665 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.557674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.558215 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.558221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.558460 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.559194 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:36.559234 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:36.561032 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:36.563103 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.563112 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.563133 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.563553 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.563875 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.563881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.563886 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.564088 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.564526 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:36.564536 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:36.564932 (+ 396us) client_negotiation.cc:769] Sending connection context
0805 23:15:36.565163 (+ 231us) client_negotiation.cc:240] Negotiation successful
0805 23:15:36.565592 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":128,"threads_started":1}
I20260805 23:15:36.568269 27464 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33421
I20260805 23:15:36.568606 27464 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:36.569235 27464 heartbeater.cc:511] Master 127.23.88.126:33421 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:36.570581 27120 ts_manager.cc:194] Registered new tserver with Master: 58ed0d69df9f4ed08c6fffa258f987f1 (127.23.88.66:43457)
I20260805 23:15:36.571287 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:58999
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:36.590977 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:36.704607 27469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:36.704947 27469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:36.705040 27469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:36.709321 27469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:36.709425 27469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:36.709523 27469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:36.715035 27469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:36.716295 27469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:36.717366 27469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:36.727854 27475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:36.727954 27478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.728052 27469 server_base.cc:1034] running on GCE node
W20260805 23:15:36.727854 27476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:36.728711 27469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:36.729321 27469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:36.730502 27469 hybrid_clock.cc:648] HybridClock initialized: now 1785971736730486 us; error 24 us; skew 500 ppm
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:36.733541 27469 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:36.734513 27469 webserver.cc:460] Webserver started at http://127.23.88.67:38009/ using document root <none> and password file <none>
I20260805 23:15:36.735148 27469 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:36.735219 27469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:36.735466 27469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:36.737782 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "232cf0aa66b4478f809d612eb347f166"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "84b0a9b972b69ab8759479e187b7ffbd"
server_key_iv: "41bd935ed2901fb1342d802211336210"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.738379 27469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "232cf0aa66b4478f809d612eb347f166"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "84b0a9b972b69ab8759479e187b7ffbd"
server_key_iv: "41bd935ed2901fb1342d802211336210"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.742300 27469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260805 23:15:36.745085 27485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.746405 27469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:36.746547 27469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "232cf0aa66b4478f809d612eb347f166"
format_stamp: "Formatted at 2026-08-05 23:15:36 on dist-test-slave-vkjp"
server_key: "84b0a9b972b69ab8759479e187b7ffbd"
server_key_iv: "41bd935ed2901fb1342d802211336210"
server_key_version: "encryptionkey@0"
I20260805 23:15:36.746676 27469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:36.765826 27469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:36.769609 27469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:36.769861 27469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:36.770531 27469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:36.771612 27469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:36.771664 27469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.771708 27469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:36.771723 27469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:36.781136 27469 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41419
I20260805 23:15:36.781158 27602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41419 every 8 connection(s)
I20260805 23:15:36.782133 27469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:36.788432 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27469
I20260805 23:15:36.788538 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:36.788846 23905 external_mini_cluster.cc:1428] Setting key ae9a8393589cb0925fbe53cbad9dd597
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:36.796989 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.784068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:38319 (local address 127.23.88.126:33421)
0805 23:15:36.784258 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:36.784262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:36.785098 (+ 836us) server_negotiation.cc:407] Connection header received
0805 23:15:36.786483 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:36.786487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:36.786550 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:36.786664 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:36.787941 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.788479 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.789750 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.790095 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.790225 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.792985 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:36.792990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:36.792992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:36.793020 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:36.794885 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.795459 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.795462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.795463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.795607 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.795944 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.795946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.795947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.796151 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:36.796229 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:36.796739 (+ 510us) server_negotiation.cc:299] Negotiation successful
0805 23:15:36.796859 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260805 23:15:36.797847 27605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.784471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33421 (local address 127.23.88.67:38319)
0805 23:15:36.784937 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:36.784980 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:36.786258 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:36.786846 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:36.786853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:36.787302 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:36.787788 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.787801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.788670 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.788673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.789616 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:36.789626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.790302 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.790307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:36.790508 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.791272 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:36.791289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:36.792778 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:36.795001 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.795005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.795018 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.795346 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.795701 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:36.795703 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:36.795705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:36.795849 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:36.796243 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:36.796248 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:36.796517 (+ 269us) client_negotiation.cc:769] Sending connection context
0805 23:15:36.796672 (+ 155us) client_negotiation.cc:240] Negotiation successful
0805 23:15:36.796937 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260805 23:15:36.799284 27603 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33421
I20260805 23:15:36.799512 27603 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:36.800006 27603 heartbeater.cc:511] Master 127.23.88.126:33421 requested a full tablet report, sending...
I20260805 23:15:36.801097 27120 ts_manager.cc:194] Registered new tserver with Master: 232cf0aa66b4478f809d612eb347f166 (127.23.88.67:41419)
I20260805 23:15:36.801712 27120 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:38319
I20260805 23:15:36.803040 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971736, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:36.830467 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:36.820878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39156 (local address 127.23.88.126:33421)
0805 23:15:36.821082 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:36.821086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:36.821182 (+ 96us) server_negotiation.cc:407] Connection header received
0805 23:15:36.821311 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:36.821314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:36.821359 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:36.821448 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:36.822196 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.822765 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.823897 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:36.824239 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:36.824341 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:36.826891 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:36.826894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:36.826896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:36.826925 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:36.828730 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.829136 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.829139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.829140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.829217 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:36.829497 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:36.829500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:36.829502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:36.829757 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:36.829918 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:36.830165 (+ 247us) server_negotiation.cc:299] Negotiation successful
0805 23:15:36.830290 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260805 23:15:36.830878 27120 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39156
I20260805 23:15:36.832007 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27189
I20260805 23:15:36.840534 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27330
I20260805 23:15:36.846832 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27469
I20260805 23:15:36.853556 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27091
2026-08-05T23:15:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2321 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27615](info): set up 2 sockets
Aug 05 23:15:36 dist-test-slave-vkjp krb5kdc[27615](info): commencing operation
krb5kdc: starting...
W20260805 23:15:37.932868 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:38.067066 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37071
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:38.183133 27630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:38.183439 27630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:38.183482 27630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:38.187580 27630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:38.187651 27630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:38.187671 27630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:38.187688 27630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:38.187703 27630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:38.193244 27630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:38.194370 27630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:38.195407 27630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:38.204147 27637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:38.204247 27630 server_base.cc:1034] running on GCE node
W20260805 23:15:38.204154 27636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:38.204154 27639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:38.205001 27630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:38.206265 27630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:38.207463 27630 hybrid_clock.cc:648] HybridClock initialized: now 1785971738207435 us; error 45 us; skew 500 ppm
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:38.214298 27630 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:38.215441 27630 webserver.cc:460] Webserver started at http://127.23.88.126:42777/ using document root <none> and password file <none>
I20260805 23:15:38.216046 27630 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:38.216148 27630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:38.216401 27630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:38.218547 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "ab2e20254deb452a85c25bca5d9cd594"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "1b3c3acda31f7ad76587595cca8e049d"
server_key_iv: "6bb4a7028fac4349e27746f3531305b5"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.219230 27630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "ab2e20254deb452a85c25bca5d9cd594"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "1b3c3acda31f7ad76587595cca8e049d"
server_key_iv: "6bb4a7028fac4349e27746f3531305b5"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.223373 27630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:38.226150 27646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.227712 27630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:38.227874 27630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "ab2e20254deb452a85c25bca5d9cd594"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "1b3c3acda31f7ad76587595cca8e049d"
server_key_iv: "6bb4a7028fac4349e27746f3531305b5"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.228009 27630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:38.255550 27630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:38.259176 27630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:38.259423 27630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:38.267385 27630 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:37071
I20260805 23:15:38.267406 27701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:37071 every 8 connection(s)
I20260805 23:15:38.268438 27630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:38.271602 27702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:38.275019 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27630
I20260805 23:15:38.275238 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:38.275590 23905 external_mini_cluster.cc:1428] Setting key 311610e7893550fd4fad7376e0a42eb7
I20260805 23:15:38.278738 27702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594: Bootstrap starting.
I20260805 23:15:38.281376 27702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:38.282243 27702 log.cc:824] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:38.284667 27702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594: No bootstrap required, opened a new log
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:38.290288 27702 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } }
I20260805 23:15:38.290647 27702 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:38.290706 27702 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2e20254deb452a85c25bca5d9cd594, State: Initialized, Role: FOLLOWER
I20260805 23:15:38.291316 27702 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } }
I20260805 23:15:38.291518 27702 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:38.291594 27702 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:38.291699 27702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:38.292152 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:38.277312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.23.88.126:37071)
0805 23:15:38.277929 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:38.277947 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:38.277989 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:15:38.278671 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:38.278690 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:38.279168 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:38.279541 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:38.280381 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.281439 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.282377 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.283047 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.283344 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:38.285914 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:38.285921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:38.285940 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:38.285977 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:38.288362 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.288865 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.288875 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.288883 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.289051 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.289361 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.289365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.289367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.289992 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:38.290206 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:38.290566 (+ 360us) server_negotiation.cc:299] Negotiation successful
0805 23:15:38.290940 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":163,"threads_started":1}
I20260805 23:15:38.293052 27702 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } }
I20260805 23:15:38.293467 27702 leader_election.cc:302] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab2e20254deb452a85c25bca5d9cd594; no voters:
I20260805 23:15:38.293893 27702 leader_election.cc:288] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:38.293982 27707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:38.294222 27707 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [term 1 LEADER]: Becoming Leader. State: Replica: ab2e20254deb452a85c25bca5d9cd594, State: Running, Role: LEADER
I20260805 23:15:38.294970 27707 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } }
I20260805 23:15:38.295414 27702 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:38.296876 27708 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab2e20254deb452a85c25bca5d9cd594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } } }
I20260805 23:15:38.297030 27708 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:38.297465 27709 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab2e20254deb452a85c25bca5d9cd594. Latest consensus state: current_term: 1 leader_uuid: "ab2e20254deb452a85c25bca5d9cd594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab2e20254deb452a85c25bca5d9cd594" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37071 } } }
I20260805 23:15:38.297585 27709 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:38.297448 27716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:38.301011 27716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:38.307742 27716 catalog_manager.cc:1347] Generated new cluster ID: 011676555bc1412dbbaf22bbdaa8094d
I20260805 23:15:38.307845 27716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:38.319486 27716 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:38.321044 27716 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:38.328150 27716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab2e20254deb452a85c25bca5d9cd594: Generated new TSK 0
I20260805 23:15:38.328958 27716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:38.372514 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:38.494503 27728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:38.494805 27728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:38.494855 27728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:38.499511 27728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:38.499585 27728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:38.499650 27728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:38.505856 27728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:38.507262 27728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:38.508445 27728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:38.519649 27734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:38.519707 27733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:38.519661 27736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:38.520008 27728 server_base.cc:1034] running on GCE node
I20260805 23:15:38.520550 27728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:38.521313 27728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:38.522532 27728 hybrid_clock.cc:648] HybridClock initialized: now 1785971738522502 us; error 43 us; skew 500 ppm
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:38.525691 27728 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:38.526751 27728 webserver.cc:460] Webserver started at http://127.23.88.65:34765/ using document root <none> and password file <none>
I20260805 23:15:38.527453 27728 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:38.527560 27728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:38.527818 27728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:38.530149 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "a1bf852597384eb19d6da5716baf29eb"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "df8de350e39d93bc624ffb675f8e1a37"
server_key_iv: "5bdc8eb1887980d9a98485db530aff02"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.530792 27728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "a1bf852597384eb19d6da5716baf29eb"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "df8de350e39d93bc624ffb675f8e1a37"
server_key_iv: "5bdc8eb1887980d9a98485db530aff02"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.535403 27728 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:38.538291 27743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.539932 27728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:38.540102 27728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "a1bf852597384eb19d6da5716baf29eb"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "df8de350e39d93bc624ffb675f8e1a37"
server_key_iv: "5bdc8eb1887980d9a98485db530aff02"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.540248 27728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:38.575671 27728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:38.579398 27728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:38.579667 27728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:38.580374 27728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:38.581336 27728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:38.581434 27728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.581537 27728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:38.581593 27728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.591506 27728 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:34093
I20260805 23:15:38.591563 27863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:34093 every 8 connection(s)
I20260805 23:15:38.592558 27728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:38.600981 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27728
I20260805 23:15:38.601141 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:38.601488 23905 external_mini_cluster.cc:1428] Setting key f5a7c97ac9b7b9964865d14d75a4301d
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:38.608647 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:38.594668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33505 (local address 127.23.88.126:37071)
0805 23:15:38.594842 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:38.594847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:38.596104 (+ 1257us) server_negotiation.cc:407] Connection header received
0805 23:15:38.597782 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:38.597785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:38.597837 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:38.597921 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:38.599224 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.599803 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.600966 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.601367 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.601488 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:38.604359 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:38.604366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:38.604372 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:38.604409 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:38.605944 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.606649 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.606652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.606654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.606735 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.607249 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.607252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.607253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.607495 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:38.607592 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:38.608372 (+ 780us) server_negotiation.cc:299] Negotiation successful
0805 23:15:38.608508 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:15:38.610006 27866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:38.595259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37071 (local address 127.23.88.65:33505)
0805 23:15:38.595820 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:38.595868 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:38.597541 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:38.598092 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:38.598099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:38.598504 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:38.599004 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:38.599019 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.599936 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.599939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:38.600819 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:38.600828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.601526 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.601528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:38.601728 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:38.602409 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:38.602427 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:38.604171 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:38.606075 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:38.606083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:38.606104 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:38.606519 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:38.606834 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:38.606840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:38.606845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:38.607058 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:38.607629 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:38.607640 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:38.608050 (+ 410us) client_negotiation.cc:769] Sending connection context
0805 23:15:38.608302 (+ 252us) client_negotiation.cc:240] Negotiation successful
0805 23:15:38.608725 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":132,"threads_started":1}
I20260805 23:15:38.611508 27864 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37071
I20260805 23:15:38.611784 27864 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:38.612339 27864 heartbeater.cc:511] Master 127.23.88.126:37071 requested a full tablet report, sending...
I20260805 23:15:38.613968 27664 ts_manager.cc:194] Registered new tserver with Master: a1bf852597384eb19d6da5716baf29eb (127.23.88.65:34093)
I20260805 23:15:38.615353 27664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:33505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:38.635821 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:38.752918 27869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:38.753216 27869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:38.753262 27869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:38.757663 27869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:38.757752 27869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:38.757817 27869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:38.763768 27869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:38.764968 27869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:38.766072 27869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:38.776098 27877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:38.776180 27875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:38.776283 27869 server_base.cc:1034] running on GCE node
W20260805 23:15:38.776134 27874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:38.776926 27869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:38.777679 27869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:38.778900 27869 hybrid_clock.cc:648] HybridClock initialized: now 1785971738778857 us; error 64 us; skew 500 ppm
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:38.782061 27869 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:38.783164 27869 webserver.cc:460] Webserver started at http://127.23.88.66:43281/ using document root <none> and password file <none>
I20260805 23:15:38.783808 27869 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:38.783877 27869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:38.784062 27869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:38.786221 27869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "5d8b281f4ab54deb8a64ef434f5e76d8"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "56dc19217b3997e3cfadd8021e9657f6"
server_key_iv: "dfa238324b9aaf5ee45721f4acb9f5bb"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.786788 27869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "5d8b281f4ab54deb8a64ef434f5e76d8"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "56dc19217b3997e3cfadd8021e9657f6"
server_key_iv: "dfa238324b9aaf5ee45721f4acb9f5bb"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.791010 27869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:15:38.793789 27884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.795301 27869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260805 23:15:38.795464 27869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "5d8b281f4ab54deb8a64ef434f5e76d8"
format_stamp: "Formatted at 2026-08-05 23:15:38 on dist-test-slave-vkjp"
server_key: "56dc19217b3997e3cfadd8021e9657f6"
server_key_iv: "dfa238324b9aaf5ee45721f4acb9f5bb"
server_key_version: "encryptionkey@0"
I20260805 23:15:38.795614 27869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:38.816124 27869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:38.819851 27869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:38.820086 27869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:38.820755 27869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:38.821672 27869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:38.821749 27869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.821822 27869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:38.821877 27869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:38.831749 27869 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:46665
I20260805 23:15:38.831787 28006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:46665 every 8 connection(s)
I20260805 23:15:38.832698 27869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:38.833602 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 27869
I20260805 23:15:38.833698 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:38.833998 23905 external_mini_cluster.cc:1428] Setting key 7cf6330b5113bdc9e587f22834bc7ddc
Aug 05 23:15:38 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971738, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:15:38.848287 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:38.834865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46623 (local address 127.23.88.126:37071)
0805 23:15:38.835014 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:38.835019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:38.836026 (+ 1007us) server_negotiation.cc:407] Connection header received
0805 23:15:38.837482 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:38.837485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:38.837543 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:38.837628 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:38.838853 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.839451 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.840625 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.840858 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.840976 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:38.843942 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:38.843946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:38.843950 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:38.843980 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:38.845773 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.846500 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.846506 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.846508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.846600 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:38.847001 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:38.847006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:38.847007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:38.847283 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:38.847409 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:38.847975 (+ 566us) server_negotiation.cc:299] Negotiation successful
0805 23:15:38.848114 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:38.849188 28009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:38.835299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37071 (local address 127.23.88.66:46623)
0805 23:15:38.835881 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:38.835926 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:38.837256 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:38.837823 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:38.837831 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:38.838207 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:38.838696 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:38.838709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.839595 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.839599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:38.840489 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:38.840499 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:38.841044 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:38.841049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:38.841224 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:38.841888 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:38.841904 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:38.843713 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:38.845938 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:38.845948 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:38.845970 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:38.846384 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:38.846707 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:38.846710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:38.846713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:38.846895 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:38.847408 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:38.847418 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:38.847711 (+ 293us) client_negotiation.cc:769] Sending connection context
0805 23:15:38.847928 (+ 217us) client_negotiation.cc:240] Negotiation successful
0805 23:15:38.848303 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":176,"threads_started":1}
I20260805 23:15:38.850456 28007 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37071
I20260805 23:15:38.850730 28007 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:38.851379 28007 heartbeater.cc:511] Master 127.23.88.126:37071 requested a full tablet report, sending...
I20260805 23:15:38.852720 27664 ts_manager.cc:194] Registered new tserver with Master: 5d8b281f4ab54deb8a64ef434f5e76d8 (127.23.88.66:46665)
I20260805 23:15:38.853451 27664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:46623
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:38.868057 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:38.984292 28012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:38.984622 28012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:38.984700 28012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:38.989027 28012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:38.989140 28012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:38.989240 28012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:38.995276 28012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:38.996694 28012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:38.997831 28012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:39.008553 28019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:39.008577 28018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:39.008741 28012 server_base.cc:1034] running on GCE node
W20260805 23:15:39.008553 28021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:39.009305 28012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:39.009994 28012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:39.011252 28012 hybrid_clock.cc:648] HybridClock initialized: now 1785971739011232 us; error 36 us; skew 500 ppm
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971739, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:39.014287 28012 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:39.015434 28012 webserver.cc:460] Webserver started at http://127.23.88.67:34445/ using document root <none> and password file <none>
I20260805 23:15:39.016041 28012 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:39.016104 28012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:39.016335 28012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:39.018518 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "b3087c00d27e4a30bbf29f995fbf0702"
format_stamp: "Formatted at 2026-08-05 23:15:39 on dist-test-slave-vkjp"
server_key: "c4b8d0c562abb30588534b6127ac537e"
server_key_iv: "6899199bc144645d139c8cf8df482e7c"
server_key_version: "encryptionkey@0"
I20260805 23:15:39.019170 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "b3087c00d27e4a30bbf29f995fbf0702"
format_stamp: "Formatted at 2026-08-05 23:15:39 on dist-test-slave-vkjp"
server_key: "c4b8d0c562abb30588534b6127ac537e"
server_key_iv: "6899199bc144645d139c8cf8df482e7c"
server_key_version: "encryptionkey@0"
I20260805 23:15:39.023247 28012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:39.026087 28028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:39.027539 28012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:39.027680 28012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "b3087c00d27e4a30bbf29f995fbf0702"
format_stamp: "Formatted at 2026-08-05 23:15:39 on dist-test-slave-vkjp"
server_key: "c4b8d0c562abb30588534b6127ac537e"
server_key_iv: "6899199bc144645d139c8cf8df482e7c"
server_key_version: "encryptionkey@0"
I20260805 23:15:39.027801 28012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:39.051036 28012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:39.054911 28012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:39.055173 28012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:39.055908 28012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:39.056838 28012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:39.056887 28012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:39.056964 28012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:39.057003 28012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:39.067685 28012 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41083
I20260805 23:15:39.067720 28148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41083 every 8 connection(s)
I20260805 23:15:39.068722 28012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:39.075857 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28012
I20260805 23:15:39.076036 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:39.076354 23905 external_mini_cluster.cc:1428] Setting key ee92faef4881992fa279614b0d867954
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971739, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:39.084371 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:39.070887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:47071 (local address 127.23.88.126:37071)
0805 23:15:39.071045 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:39.071049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:39.071993 (+ 944us) server_negotiation.cc:407] Connection header received
0805 23:15:39.073494 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:39.073497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:39.073564 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:39.073690 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:39.075017 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.075653 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.077236 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.077566 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.077708 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:39.080554 (+ 2846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:39.080558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:39.080560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:39.080586 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:39.082205 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.082732 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.082734 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.082736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.082816 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.083215 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.083220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.083226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.083430 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:39.083509 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:39.084133 (+ 624us) server_negotiation.cc:299] Negotiation successful
0805 23:15:39.084241 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:15:39.085311 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:39.071246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37071 (local address 127.23.88.67:47071)
0805 23:15:39.071784 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:39.071832 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:39.073241 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:39.073896 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:39.073904 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:39.074312 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:39.074837 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:39.074853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.075821 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.075824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:39.077103 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:39.077113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.077790 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.077795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:39.078008 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:39.078727 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:39.078746 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:39.080345 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:39.082324 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:39.082330 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:39.082345 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:39.082632 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:39.082907 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:39.082909 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:39.082910 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:39.083052 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:39.083572 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:39.083579 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:39.083874 (+ 295us) client_negotiation.cc:769] Sending connection context
0805 23:15:39.084045 (+ 171us) client_negotiation.cc:240] Negotiation successful
0805 23:15:39.084354 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":128,"threads_started":1}
I20260805 23:15:39.086653 28149 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37071
I20260805 23:15:39.086946 28149 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:39.087528 28149 heartbeater.cc:511] Master 127.23.88.126:37071 requested a full tablet report, sending...
I20260805 23:15:39.088768 27664 ts_manager.cc:194] Registered new tserver with Master: b3087c00d27e4a30bbf29f995fbf0702 (127.23.88.67:41083)
I20260805 23:15:39.089469 27664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:47071
I20260805 23:15:39.090981 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[27615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:39.120134 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:39.109203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.23.88.65:34093)
0805 23:15:39.109577 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:39.109603 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:39.109629 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:15:39.109681 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:39.109686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:39.109894 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:39.110026 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:39.110923 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.111772 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.112681 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.113080 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.113173 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:39.115904 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:39.115916 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:39.115932 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:39.115963 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:39.118022 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.118536 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.118542 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.118543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.118635 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.118965 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.118967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.118969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.119368 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:39.119554 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:39.119808 (+ 254us) server_negotiation.cc:299] Negotiation successful
0805 23:15:39.119957 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":101,"threads_started":1}
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[27615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:39.132406 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:39.123598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.23.88.126:37071)
0805 23:15:39.123753 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:39.123759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:39.123826 (+ 67us) server_negotiation.cc:407] Connection header received
0805 23:15:39.123988 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:39.123990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:39.124046 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:39.124137 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:39.124854 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.125400 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.126214 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:39.126450 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:39.126565 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:39.129191 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:39.129194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:39.129196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:39.129225 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:39.130497 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.131000 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.131003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.131004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.131129 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:39.131436 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:39.131439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:39.131441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:39.131689 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:39.131843 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:39.132099 (+ 256us) server_negotiation.cc:299] Negotiation successful
0805 23:15:39.132226 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260805 23:15:39.132881 27664 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55840
I20260805 23:15:39.133566 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27728
I20260805 23:15:39.141192 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27869
I20260805 23:15:39.148270 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28012
I20260805 23:15:39.155454 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 27630
2026-08-05T23:15:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2302 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[28161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[28161](info): set up 2 sockets
Aug 05 23:15:39 dist-test-slave-vkjp krb5kdc[28161](info): commencing operation
krb5kdc: starting...
W20260805 23:15:40.277096 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:40 dist-test-slave-vkjp krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:40.431339 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33873
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:40.554611 28174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:40.554960 28174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:40.555047 28174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:40.559541 28174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:40.559640 28174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:40.559679 28174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:40.559736 28174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:40.559774 28174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:40.565442 28174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:40.566668 28174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:40.567791 28174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:40.577085 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:40.577076 28180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:40.577134 28174 server_base.cc:1034] running on GCE node
W20260805 23:15:40.577076 28182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:40.578176 28174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:40.579538 28174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:40.580781 28174 hybrid_clock.cc:648] HybridClock initialized: now 1785971740580760 us; error 37 us; skew 500 ppm
Aug 05 23:15:40 dist-test-slave-vkjp krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:40.583969 28174 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:40.584820 28174 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:40.584926 28174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:40.585186 28174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:40.587592 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "6ecdf0d21a2447c5bb73ffe0253d3490"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "af5ab45a37636a233ba88643d2863929"
server_key_iv: "865268686e26620e1dec538da68c35be"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.588215 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "6ecdf0d21a2447c5bb73ffe0253d3490"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "af5ab45a37636a233ba88643d2863929"
server_key_iv: "865268686e26620e1dec538da68c35be"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.592293 28174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:40.595002 28188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:40.596457 28174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:40.596626 28174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "6ecdf0d21a2447c5bb73ffe0253d3490"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "af5ab45a37636a233ba88643d2863929"
server_key_iv: "865268686e26620e1dec538da68c35be"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.596758 28174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:40.611819 28174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:40.614979 28174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:40.615255 28174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:40.622843 28174 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:33873
I20260805 23:15:40.623172 28244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:33873 every 8 connection(s)
I20260805 23:15:40.623672 28174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:40.626552 28245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:40.629020 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28174
I20260805 23:15:40.629218 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:40.629540 23905 external_mini_cluster.cc:1428] Setting key 85709e701d4940091182ac69f8ac1303
I20260805 23:15:40.632944 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: Bootstrap starting.
I20260805 23:15:40.635716 28245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:40.636633 28245 log.cc:824] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:40.638643 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: No bootstrap required, opened a new log
Aug 05 23:15:40 dist-test-slave-vkjp krb5kdc[28161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:40.642477 28245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } }
I20260805 23:15:40.642752 28245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:40.642838 28245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ecdf0d21a2447c5bb73ffe0253d3490, State: Initialized, Role: FOLLOWER
I20260805 23:15:40.643405 28245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } }
I20260805 23:15:40.643589 28245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:40.643661 28245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:40.643765 28245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:40.644908 28245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } }
I20260805 23:15:40.645303 28245 leader_election.cc:302] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ecdf0d21a2447c5bb73ffe0253d3490; no voters:
I20260805 23:15:40.645666 28245 leader_election.cc:288] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:40.645814 28250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:40.646045 28250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [term 1 LEADER]: Becoming Leader. State: Replica: 6ecdf0d21a2447c5bb73ffe0253d3490, State: Running, Role: LEADER
I20260805 23:15:40.646493 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:40.631162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44322 (local address 127.23.88.126:33873)
0805 23:15:40.631697 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:40.631714 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:40.631756 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:15:40.632424 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:40.632441 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:40.632793 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:40.633136 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:40.634050 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.635179 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.636447 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.637071 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.637276 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:40.640216 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:40.640222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:40.640236 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:40.640267 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:40.642952 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:40.643507 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:40.643513 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:40.643517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:40.643690 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:40.644001 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:40.644004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:40.644005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:40.644447 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:40.644621 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:40.644931 (+ 310us) server_negotiation.cc:299] Negotiation successful
0805 23:15:40.645258 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":177,"threads_started":1}
I20260805 23:15:40.646450 28250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } }
I20260805 23:15:40.647019 28245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:40.647796 28250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ecdf0d21a2447c5bb73ffe0253d3490. Latest consensus state: current_term: 1 leader_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } } }
I20260805 23:15:40.647943 28250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:40.648459 28251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ecdf0d21a2447c5bb73ffe0253d3490" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33873 } } }
I20260805 23:15:40.648661 28251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:40.648978 28261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:40.652364 28261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:40.658371 28261 catalog_manager.cc:1347] Generated new cluster ID: 21efc3384b0b479cb481870de63a15a2
I20260805 23:15:40.658468 28261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:40.678089 28261 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:40.679214 28261 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:40.689667 28261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: Generated new TSK 0
I20260805 23:15:40.690448 28261 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:40.726615 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:40.849372 28272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:40.849685 28272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:40.849733 28272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:40.853994 28272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:40.854074 28272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:40.854143 28272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:40.859879 28272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:40.861013 28272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:40.862038 28272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:40.872982 28279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:40.872984 28280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:40.873025 28272 server_base.cc:1034] running on GCE node
W20260805 23:15:40.872982 28282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:40.873837 28272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:40.874552 28272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:40.875752 28272 hybrid_clock.cc:648] HybridClock initialized: now 1785971740875736 us; error 34 us; skew 500 ppm
Aug 05 23:15:40 dist-test-slave-vkjp krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:40.878926 28272 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:40.879812 28272 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:40.879921 28272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:40.880165 28272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:40.882448 28272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "2d2becbf17394611824f9e45c8136eaf"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "7a8d0a4ec9abec0882af1d6e03e3cb6c"
server_key_iv: "64dba83051fb7c3bbf63bebc1a25a977"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.883064 28272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "2d2becbf17394611824f9e45c8136eaf"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "7a8d0a4ec9abec0882af1d6e03e3cb6c"
server_key_iv: "64dba83051fb7c3bbf63bebc1a25a977"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.887071 28272 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:15:40.889976 28288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:40.891431 28272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:40.891582 28272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "2d2becbf17394611824f9e45c8136eaf"
format_stamp: "Formatted at 2026-08-05 23:15:40 on dist-test-slave-vkjp"
server_key: "7a8d0a4ec9abec0882af1d6e03e3cb6c"
server_key_iv: "64dba83051fb7c3bbf63bebc1a25a977"
server_key_version: "encryptionkey@0"
I20260805 23:15:40.891716 28272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:40.905853 28272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:40.909546 28272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:40.909799 28272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:40.910454 28272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:40.911409 28272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:40.911484 28272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:40.911580 28272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:40.911630 28272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:40.921340 28272 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:34361
I20260805 23:15:40.921357 28408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:34361 every 8 connection(s)
I20260805 23:15:40.922016 28272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:40.924443 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28272
I20260805 23:15:40.924582 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:40.924919 23905 external_mini_cluster.cc:1428] Setting key 50a72064e381c622a885374429c9e146
Aug 05 23:15:40 dist-test-slave-vkjp krb5kdc[28161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971740, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:40.939375 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:40.923890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:48101 (local address 127.23.88.126:33873)
0805 23:15:40.924046 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:40.924050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:40.925074 (+ 1024us) server_negotiation.cc:407] Connection header received
0805 23:15:40.926786 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:40.926789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:40.926841 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:40.926906 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:40.928700 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.929626 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.931520 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.931757 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.931848 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:40.935069 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:40.935072 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:40.935074 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:40.935151 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:40.936759 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:40.937373 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:40.937377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:40.937379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:40.937484 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:40.937882 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:40.937885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:40.937892 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:40.938164 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:40.938258 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:40.938896 (+ 638us) server_negotiation.cc:299] Negotiation successful
0805 23:15:40.939121 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:15:40.940368 28412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:40.924299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33873 (local address 127.23.88.65:48101)
0805 23:15:40.924883 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:40.924956 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:40.926552 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:40.927158 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:40.927169 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:40.927730 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:40.928494 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:40.928514 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.929787 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.929792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:40.931367 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:40.931383 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:40.931951 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:40.931958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:40.932200 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:40.932880 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:40.932919 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:40.934847 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:40.936881 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:40.936886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:40.936943 (+ 57us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:40.937231 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:40.937589 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:40.937591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:40.937592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:40.937739 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:40.938276 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:40.938282 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:40.938557 (+ 275us) client_negotiation.cc:769] Sending connection context
0805 23:15:40.938765 (+ 208us) client_negotiation.cc:240] Negotiation successful
0805 23:15:40.939174 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":154,"threads_started":1}
I20260805 23:15:40.941949 28409 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33873
I20260805 23:15:40.942206 28409 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:40.942798 28409 heartbeater.cc:511] Master 127.23.88.126:33873 requested a full tablet report, sending...
I20260805 23:15:40.944550 28207 ts_manager.cc:194] Registered new tserver with Master: 2d2becbf17394611824f9e45c8136eaf (127.23.88.65:34361)
I20260805 23:15:40.945820 28207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:48101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:40.958560 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:41.073513 28415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:41.073808 28415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:41.073860 28415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:41.078110 28415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:41.078197 28415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:41.078263 28415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:41.084059 28415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:41.085206 28415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:41.086257 28415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:41.096312 28420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:41.096374 28424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:41.096267 28421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:41.096700 28415 server_base.cc:1034] running on GCE node
I20260805 23:15:41.097204 28415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:41.097967 28415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:41.099167 28415 hybrid_clock.cc:648] HybridClock initialized: now 1785971741099125 us; error 44 us; skew 500 ppm
Aug 05 23:15:41 dist-test-slave-vkjp krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:41.102562 28415 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:41.103484 28415 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:41.103586 28415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:41.103837 28415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:41.106065 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "10d232588d8af64f3c28e8c62d823506"
server_key_iv: "55450f3622f004127e162626b328ba27"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.106693 28415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "10d232588d8af64f3c28e8c62d823506"
server_key_iv: "55450f3622f004127e162626b328ba27"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.110726 28415 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260805 23:15:41.113608 28430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.115031 28415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:41.115267 28415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "10d232588d8af64f3c28e8c62d823506"
server_key_iv: "55450f3622f004127e162626b328ba27"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.115404 28415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:41.134490 28415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:41.137679 28415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:41.137907 28415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:41.138520 28415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:41.139446 28415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:41.139521 28415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.139599 28415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:41.139653 28415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.149467 28415 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:43497
I20260805 23:15:41.149533 28547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:43497 every 8 connection(s)
I20260805 23:15:41.150147 28415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:41.156069 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28415
I20260805 23:15:41.156225 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:41.156525 23905 external_mini_cluster.cc:1428] Setting key 3af81872a7a0dc651602c2ec07a81f2c
Aug 05 23:15:41 dist-test-slave-vkjp krb5kdc[28161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:41.166137 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.152029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:41363 (local address 127.23.88.126:33873)
0805 23:15:41.152207 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.152211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.153085 (+ 874us) server_negotiation.cc:407] Connection header received
0805 23:15:41.154578 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.154582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.154647 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.154768 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:41.156164 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.156922 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.158190 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.158446 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.158537 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.161612 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:41.161620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:41.161627 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:41.161666 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:41.163597 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.164170 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.164175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.164179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.164292 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.164737 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.164744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.164749 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.165060 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:41.165194 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.165756 (+ 562us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.165912 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:15:41.166947 28550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.152417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33873 (local address 127.23.88.66:41363)
0805 23:15:41.152913 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.152963 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.154310 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.154962 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.154970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.155458 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:41.155998 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.156013 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.157078 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.157082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.158054 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.158067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.158623 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.158627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.158897 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.159641 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:41.159669 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:41.161423 (+ 1754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:41.163728 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:41.163733 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:41.163747 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:41.164049 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:41.164453 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:41.164456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:41.164457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:41.164620 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:41.165191 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:41.165196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:41.165478 (+ 282us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.165647 (+ 169us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.165936 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:41.168362 28548 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33873
I20260805 23:15:41.168599 28548 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:41.169085 28548 heartbeater.cc:511] Master 127.23.88.126:33873 requested a full tablet report, sending...
I20260805 23:15:41.170235 28207 ts_manager.cc:194] Registered new tserver with Master: c32338a8fb5a4e5bba6aabaab6d5c62a (127.23.88.66:43497)
I20260805 23:15:41.170843 28207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:41363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:41.188400 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:41.304502 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:41.304780 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:41.304826 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:41.308962 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:41.309032 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:41.309095 28553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:41.314545 28553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:41.315795 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:41.316887 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:41.326887 28558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:41.326951 28561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:41.326858 28559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:41.327265 28553 server_base.cc:1034] running on GCE node
I20260805 23:15:41.327759 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:41.328589 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:41.329826 28553 hybrid_clock.cc:648] HybridClock initialized: now 1785971741329796 us; error 41 us; skew 500 ppm
Aug 05 23:15:41 dist-test-slave-vkjp krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:41.333042 28553 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:41.333850 28553 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:41.333922 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:41.334107 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:41.336519 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "c4c0ac30ca624780b13c963e644a6798"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "cd1e7cf62ae83f233b1edd0dfa8d131f"
server_key_iv: "bc9a6ec9e4d44a27e713f7c8920ce5ed"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.337205 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "c4c0ac30ca624780b13c963e644a6798"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "cd1e7cf62ae83f233b1edd0dfa8d131f"
server_key_iv: "bc9a6ec9e4d44a27e713f7c8920ce5ed"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.341531 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:41.344290 28568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.345775 28553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:41.345963 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "c4c0ac30ca624780b13c963e644a6798"
format_stamp: "Formatted at 2026-08-05 23:15:41 on dist-test-slave-vkjp"
server_key: "cd1e7cf62ae83f233b1edd0dfa8d131f"
server_key_iv: "bc9a6ec9e4d44a27e713f7c8920ce5ed"
server_key_version: "encryptionkey@0"
I20260805 23:15:41.346098 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:41.375273 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:41.378912 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:41.379165 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:41.379820 28553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:41.380736 28553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:41.380810 28553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.380884 28553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:41.380936 28553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:41.390657 28553 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:39161
I20260805 23:15:41.390674 28687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:39161 every 8 connection(s)
I20260805 23:15:41.391379 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:41.396065 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28553
I20260805 23:15:41.396247 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:41.396552 23905 external_mini_cluster.cc:1428] Setting key e73456dc00c215091134f727d0a73935
Aug 05 23:15:41 dist-test-slave-vkjp krb5kdc[28161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971741, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:41.410425 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.393411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:35551 (local address 127.23.88.126:33873)
0805 23:15:41.393567 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.393571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.394483 (+ 912us) server_negotiation.cc:407] Connection header received
0805 23:15:41.396154 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.396157 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.396246 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.396363 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:41.398306 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.398948 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.400326 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.400690 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.400832 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.405004 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:41.405013 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:41.405020 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:41.405063 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:41.407309 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.407985 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.407991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.408018 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.408155 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.408674 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.408681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.408686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.409039 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:41.409174 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.409963 (+ 789us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.410243 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:15:41.411497 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.393828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33873 (local address 127.23.88.67:35551)
0805 23:15:41.394317 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.394362 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.395765 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.396610 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.396619 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.397276 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:41.398136 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.398163 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.399110 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.399114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.400162 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.400174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.401090 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.401100 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.401363 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.402386 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:41.402418 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:41.404758 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:41.407473 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:41.407479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:41.407527 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:41.407833 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:41.408302 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:41.408309 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:41.408312 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:41.408529 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:41.409195 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:41.409203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:41.409616 (+ 413us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.409906 (+ 290us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.410291 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":142,"threads_started":1}
I20260805 23:15:41.412940 28688 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33873
I20260805 23:15:41.413209 28688 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:41.413726 28688 heartbeater.cc:511] Master 127.23.88.126:33873 requested a full tablet report, sending...
I20260805 23:15:41.414835 28207 ts_manager.cc:194] Registered new tserver with Master: c4c0ac30ca624780b13c963e644a6798 (127.23.88.67:39161)
I20260805 23:15:41.415530 28207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:35551
I20260805 23:15:41.423672 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:15:41.433236 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.425761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.23.88.126:33873)
0805 23:15:41.425930 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.425934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.426086 (+ 152us) server_negotiation.cc:407] Connection header received
0805 23:15:41.426182 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.426184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.426233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.426325 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:41.427223 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.427764 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.428610 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.428820 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.428916 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.430337 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:41.430339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:41.430341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:41.430362 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:41.431677 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.432076 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.432078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.432080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.432154 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:41.432394 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:41.432396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:41.432398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:41.432616 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:41.432719 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.432969 (+ 250us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.433092 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:15:41.436335 28207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:41.439535 28207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:41.454938 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.448837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39161 (local address 127.0.0.1:59004)
0805 23:15:41.449642 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.449680 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.449840 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.450797 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.450801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.450820 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:41.451003 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.451011 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452265 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.452272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.453491 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.453501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.454314 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.454318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.454449 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.454493 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.454631 (+ 138us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.454761 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260805 23:15:41.454947 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.449116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59004 (local address 127.23.88.67:39161)
0805 23:15:41.449486 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.449499 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.450306 (+ 807us) server_negotiation.cc:407] Connection header received
0805 23:15:41.450395 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.450400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.450618 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.451183 (+ 565us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:41.451340 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452133 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.453619 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.454182 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.454312 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.454476 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.454665 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.454770 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":151,"threads_started":1}
I20260805 23:15:41.456254 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.448938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.23.88.65:34361)
0805 23:15:41.450160 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.450167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.450269 (+ 102us) server_negotiation.cc:407] Connection header received
0805 23:15:41.450541 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.450548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.450787 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.450966 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:41.451175 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452585 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.454353 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.455323 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.455467 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.455625 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.455996 (+ 371us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.456113 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1095,"thread_start_us":109,"threads_started":1}
I20260805 23:15:41.456329 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.449979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.23.88.66:43497)
0805 23:15:41.450731 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.450745 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.450773 (+ 28us) server_negotiation.cc:407] Connection header received
0805 23:15:41.450861 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.450869 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.451050 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.451290 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:41.451524 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452801 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.454444 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.455332 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.455467 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.455625 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.456075 (+ 450us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.456174 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":151,"threads_started":1}
I20260805 23:15:41.456650 28701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.449800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:43497 (local address 127.0.0.1:56106)
0805 23:15:41.450445 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.450578 (+ 133us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.450667 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.451238 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.451241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.451257 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:41.451401 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.451407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452975 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.452979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.454279 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.454295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.455480 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.455484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.455586 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.455609 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.455708 (+ 99us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.456514 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":112,"threads_started":1}
I20260805 23:15:41.456977 28703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.448837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:34361 (local address 127.0.0.1:49536)
0805 23:15:41.450156 (+ 1319us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.450172 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.450337 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.450937 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.450940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.450952 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:41.451037 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.451046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.452761 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.452765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.454160 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.454173 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.455731 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.455735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.455819 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.455840 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.455957 (+ 117us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.456758 (+ 801us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1248,"mutex_wait_us":210,"thread_start_us":105,"threads_started":1}
I20260805 23:15:41.458803 28619 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c (DEFAULT_TABLE table=test-table [id=d0323e580a574af29ee1426f181f05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:41.459580 28480 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c (DEFAULT_TABLE table=test-table [id=d0323e580a574af29ee1426f181f05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:41.460083 28619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:41.459884 28341 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c (DEFAULT_TABLE table=test-table [id=d0323e580a574af29ee1426f181f05eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:41.460569 28480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:41.461073 28341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:41.467132 28705 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Bootstrap starting.
I20260805 23:15:41.468571 28707 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Bootstrap starting.
I20260805 23:15:41.468757 28706 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Bootstrap starting.
I20260805 23:15:41.470989 28707 tablet_bootstrap.cc:654] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:41.470989 28705 tablet_bootstrap.cc:654] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:41.471004 28706 tablet_bootstrap.cc:654] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:41.472093 28706 log.cc:824] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:41.472112 28707 log.cc:824] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:41.472347 28705 log.cc:824] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:41.474327 28706 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: No bootstrap required, opened a new log
I20260805 23:15:41.474565 28706 ts_tablet_manager.cc:1397] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:15:41.474593 28707 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: No bootstrap required, opened a new log
I20260805 23:15:41.474593 28705 tablet_bootstrap.cc:492] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: No bootstrap required, opened a new log
I20260805 23:15:41.474768 28707 ts_tablet_manager.cc:1397] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:15:41.474779 28705 ts_tablet_manager.cc:1397] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260805 23:15:41.477615 28706 raft_consensus.cc:348] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.477885 28706 raft_consensus.cc:374] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:41.477960 28706 raft_consensus.cc:729] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c0ac30ca624780b13c963e644a6798, State: Initialized, Role: FOLLOWER
I20260805 23:15:41.478510 28706 consensus_queue.cc:260] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.478760 28705 raft_consensus.cc:348] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.479130 28705 raft_consensus.cc:374] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:41.479233 28705 raft_consensus.cc:729] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c32338a8fb5a4e5bba6aabaab6d5c62a, State: Initialized, Role: FOLLOWER
I20260805 23:15:41.479120 28707 raft_consensus.cc:348] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.479437 28707 raft_consensus.cc:374] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:41.479578 28707 raft_consensus.cc:729] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2becbf17394611824f9e45c8136eaf, State: Initialized, Role: FOLLOWER
I20260805 23:15:41.479751 28688 heartbeater.cc:503] Master 127.23.88.126:33873 was elected leader, sending a full tablet report...
I20260805 23:15:41.479771 28706 ts_tablet_manager.cc:1428] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:15:41.480026 28707 consensus_queue.cc:260] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.480741 28409 heartbeater.cc:503] Master 127.23.88.126:33873 was elected leader, sending a full tablet report...
I20260805 23:15:41.480872 28705 consensus_queue.cc:260] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.481003 28707 ts_tablet_manager.cc:1428] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260805 23:15:41.481844 28705 ts_tablet_manager.cc:1428] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:15:41.481937 28548 heartbeater.cc:503] Master 127.23.88.126:33873 was elected leader, sending a full tablet report...
W20260805 23:15:41.642350 28689 tablet.cc:2369] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:41.651314 28549 tablet.cc:2369] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:41.671464 28711 raft_consensus.cc:482] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:41.671680 28711 raft_consensus.cc:504] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.672813 28711 leader_election.cc:288] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d2becbf17394611824f9e45c8136eaf (127.23.88.65:34361), c32338a8fb5a4e5bba6aabaab6d5c62a (127.23.88.66:43497)
W20260805 23:15:41.673482 28410 tablet.cc:2369] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:41.677165 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.673225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:34361 (local address 127.23.88.67:36959)
0805 23:15:41.673555 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.673575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.673702 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.674148 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.674151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.674171 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:41.674308 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.674314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.675054 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.675057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.675925 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.675935 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.676643 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.676647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.676805 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.676824 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.676938 (+ 114us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.677021 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"spinlock_wait_cycles":6912}
I20260805 23:15:41.677165 28700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.673363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:36959 (local address 127.23.88.65:34361)
0805 23:15:41.673557 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.673577 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.673793 (+ 216us) server_negotiation.cc:407] Connection header received
0805 23:15:41.673854 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.673857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.673934 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.674007 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:41.674435 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.674930 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.676060 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.676501 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.676643 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.676707 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.676957 (+ 250us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.677038 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:15:41.677924 28704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.673830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:41375 (local address 127.23.88.66:43497)
0805 23:15:41.674035 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.674040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.674191 (+ 151us) server_negotiation.cc:407] Connection header received
0805 23:15:41.674377 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.674380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.674448 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.674545 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:41.674846 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.675496 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.676725 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.677183 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.677359 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.677432 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.677744 (+ 312us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.677811 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260805 23:15:41.677959 28714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.673662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:43497 (local address 127.23.88.67:41375)
0805 23:15:41.674072 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.674090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:41.674257 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:41.674555 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:41.674558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:41.674577 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:41.674712 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.674719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.675648 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.675651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.676570 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:41.676584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.677462 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.677466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:41.677580 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.677622 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 23:15:41.677717 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:15:41.677802 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"spinlock_wait_cycles":29056,"thread_start_us":142,"threads_started":1}
I20260805 23:15:41.677994 28361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1917f6e0f5454a83c5b3f99cd66a3c" candidate_uuid: "c4c0ac30ca624780b13c963e644a6798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2becbf17394611824f9e45c8136eaf" is_pre_election: true
I20260805 23:15:41.678390 28361 raft_consensus.cc:2454] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c0ac30ca624780b13c963e644a6798 in term 0.
I20260805 23:15:41.678665 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1917f6e0f5454a83c5b3f99cd66a3c" candidate_uuid: "c4c0ac30ca624780b13c963e644a6798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" is_pre_election: true
I20260805 23:15:41.678983 28500 raft_consensus.cc:2454] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c0ac30ca624780b13c963e644a6798 in term 0.
I20260805 23:15:41.679028 28573 leader_election.cc:302] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2becbf17394611824f9e45c8136eaf, c4c0ac30ca624780b13c963e644a6798; no voters:
I20260805 23:15:41.679425 28711 raft_consensus.cc:2781] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:41.679513 28711 raft_consensus.cc:482] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:41.679545 28711 raft_consensus.cc:3037] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:41.724045 28711 raft_consensus.cc:504] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.724653 28711 leader_election.cc:288] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [CANDIDATE]: Term 1 election: Requested vote from peers 2d2becbf17394611824f9e45c8136eaf (127.23.88.65:34361), c32338a8fb5a4e5bba6aabaab6d5c62a (127.23.88.66:43497)
I20260805 23:15:41.725297 28361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1917f6e0f5454a83c5b3f99cd66a3c" candidate_uuid: "c4c0ac30ca624780b13c963e644a6798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d2becbf17394611824f9e45c8136eaf"
I20260805 23:15:41.725432 28500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1917f6e0f5454a83c5b3f99cd66a3c" candidate_uuid: "c4c0ac30ca624780b13c963e644a6798" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a"
I20260805 23:15:41.725509 28361 raft_consensus.cc:3037] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:41.725610 28500 raft_consensus.cc:3037] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:41.726778 28361 raft_consensus.cc:2454] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c0ac30ca624780b13c963e644a6798 in term 1.
I20260805 23:15:41.727242 28573 leader_election.cc:302] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2becbf17394611824f9e45c8136eaf, c4c0ac30ca624780b13c963e644a6798; no voters:
I20260805 23:15:41.727351 28500 raft_consensus.cc:2454] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c0ac30ca624780b13c963e644a6798 in term 1.
I20260805 23:15:41.727481 28711 raft_consensus.cc:2781] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:41.727926 28711 raft_consensus.cc:686] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 1 LEADER]: Becoming Leader. State: Replica: c4c0ac30ca624780b13c963e644a6798, State: Running, Role: LEADER
I20260805 23:15:41.728330 28711 consensus_queue.cc:237] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } }
I20260805 23:15:41.731859 28207 catalog_manager.cc:5495] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4c0ac30ca624780b13c963e644a6798 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "c4c0ac30ca624780b13c963e644a6798" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c0ac30ca624780b13c963e644a6798" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:41.757557 28702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.753828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.23.88.67:39161)
0805 23:15:41.753972 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:41.753977 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:41.754009 (+ 32us) server_negotiation.cc:407] Connection header received
0805 23:15:41.754124 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:41.754127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:41.754183 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:41.754292 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:41.754529 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.755049 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.756103 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:41.756317 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:41.756431 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:41.756731 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:41.757175 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:41.757266 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:41.757378 (+ 112us) server_negotiation.cc:299] Negotiation successful
0805 23:15:41.757436 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:15:41.764464 28500 raft_consensus.cc:1261] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 1 FOLLOWER]: Refusing update from remote peer c4c0ac30ca624780b13c963e644a6798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:41.764464 28361 raft_consensus.cc:1261] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 1 FOLLOWER]: Refusing update from remote peer c4c0ac30ca624780b13c963e644a6798: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:41.765290 28711 consensus_queue.cc:1035] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c32338a8fb5a4e5bba6aabaab6d5c62a" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 43497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:41.765522 28715 consensus_queue.cc:1035] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2becbf17394611824f9e45c8136eaf" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:41.775625 28721 mvcc.cc:205] Tried to move back new op lower bound from 7315340254258528256 to 7315340254122065920. Current Snapshot: MvccSnapshot[applied={T|T < 7315340254258528256}]
I20260805 23:15:41.785307 28207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:41.785559 28207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:41.788491 28207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ecdf0d21a2447c5bb73ffe0253d3490: Sending DeleteTablet for 3 replicas of tablet 5c1917f6e0f5454a83c5b3f99cd66a3c
I20260805 23:15:41.789530 28619 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:41 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:59004
I20260805 23:15:41.789582 28480 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:41 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:56106
I20260805 23:15:41.789716 28341 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:41 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:49536
I20260805 23:15:41.790218 28732 tablet_replica.cc:330] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf: stopping tablet replica
I20260805 23:15:41.790468 28730 tablet_replica.cc:330] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: stopping tablet replica
I20260805 23:15:41.790673 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28272
I20260805 23:15:41.790771 28730 raft_consensus.cc:2229] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:41.790782 28732 raft_consensus.cc:2229] T 5c1917f6e0f5454a83c5b3f99cd66a3c P 2d2becbf17394611824f9e45c8136eaf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:41.791322 28729 tablet_replica.cc:330] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: stopping tablet replica
I20260805 23:15:41.791642 28729 raft_consensus.cc:2229] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:41.792021 28729 raft_consensus.cc:2258] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:41.792389 28730 raft_consensus.cc:2258] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:41.794147 28730 ts_tablet_manager.cc:1905] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:41.797235 28729 ts_tablet_manager.cc:1905] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c32338a8fb5a4e5bba6aabaab6d5c62a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:15:41.803506 28193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:34361 (error 108)
I20260805 23:15:41.803591 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28415
I20260805 23:15:41.806229 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.805841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:34361 (local address 127.0.0.1:49540)
0805 23:15:41.806053 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.806125 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:34361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260805 23:15:41.806473 28193 catalog_manager.cc:4584] TS 2d2becbf17394611824f9e45c8136eaf (127.23.88.65:34361): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c1917f6e0f5454a83c5b3f99cd66a3c: Network error: Client connection negotiation failed: client connection to 127.23.88.65:34361: connect: Connection refused (error 111)
I20260805 23:15:41.832608 28730 ts_tablet_manager.cc:1918] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:41.832867 28730 log.cc:1196] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/5c1917f6e0f5454a83c5b3f99cd66a3c
I20260805 23:15:41.833295 28730 ts_tablet_manager.cc:1939] T 5c1917f6e0f5454a83c5b3f99cd66a3c P c4c0ac30ca624780b13c963e644a6798: Deleting consensus metadata
I20260805 23:15:41.834184 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28553
I20260805 23:15:41.835892 28699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:41.834424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:43497 (local address 127.0.0.1:56120)
0805 23:15:41.835694 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:41.835771 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:43497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1134}
I20260805 23:15:41.855973 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28174
2026-08-05T23:15:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2700 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-05T23:15:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:41Z Disabled control of system clock
I20260805 23:15:41.889786 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36015
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:36015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 23:15:42.017699 28737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:42.018057 28737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:42.018138 28737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:42.022516 28737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:42.022636 28737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:42.022667 28737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:42.022707 28737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:42.022743 28737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:42.028220 28737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:36015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:42.030166 28737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:42.031800 28737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:42.041306 28743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.041304 28742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.041306 28745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:42.041568 28737 server_base.cc:1034] running on GCE node
I20260805 23:15:42.042407 28737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:42.043948 28737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:42.045720 28737 hybrid_clock.cc:648] HybridClock initialized: now 1785971742045683 us; error 55 us; skew 500 ppm
I20260805 23:15:42.047959 28737 webserver.cc:460] Webserver started at http://127.23.88.126:35511/ using document root <none> and password file <none>
I20260805 23:15:42.048600 28737 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:42.048667 28737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:42.048847 28737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:42.051126 28737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "7bf6e1a4b900440cbe4989be1460b2f8"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "46914c9c93b33fe981e7b9e8f82a8e1f"
server_key_iv: "92c5308b38208bc3fdb2106fe7f63c0b"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.051777 28737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "7bf6e1a4b900440cbe4989be1460b2f8"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "46914c9c93b33fe981e7b9e8f82a8e1f"
server_key_iv: "92c5308b38208bc3fdb2106fe7f63c0b"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.056056 28737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:42.058924 28751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.060423 28737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:15:42.060571 28737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "7bf6e1a4b900440cbe4989be1460b2f8"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "46914c9c93b33fe981e7b9e8f82a8e1f"
server_key_iv: "92c5308b38208bc3fdb2106fe7f63c0b"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.060709 28737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:42.066612 28737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:42.067499 28737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:42.067718 28737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:42.075784 28737 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:36015
I20260805 23:15:42.075802 28806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:36015 every 8 connection(s)
I20260805 23:15:42.076984 28737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:42.080624 28807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:42.086769 28807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: Bootstrap starting.
I20260805 23:15:42.087172 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28737
I20260805 23:15:42.087301 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:42.087594 23905 external_mini_cluster.cc:1428] Setting key 6cbb66b6b99915c3abcd93c2d200a435
I20260805 23:15:42.089785 28807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:42.090605 28807 log.cc:824] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:42.093160 28807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: No bootstrap required, opened a new log
I20260805 23:15:42.093892 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.089031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.23.88.126:36015)
0805 23:15:42.089782 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.089798 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.089836 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:15:42.090633 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.090652 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.091195 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.091593 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.092435 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.092441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.092452 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.092462 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.092516 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.092533 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.092625 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.092759 (+ 134us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.093061 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":155,"threads_started":1}
I20260805 23:15:42.097990 28807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } }
I20260805 23:15:42.098275 28807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:42.098361 28807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf6e1a4b900440cbe4989be1460b2f8, State: Initialized, Role: FOLLOWER
I20260805 23:15:42.098805 28807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } }
I20260805 23:15:42.098964 28807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:42.099061 28807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:42.099196 28807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:42.122889 28807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } }
I20260805 23:15:42.123551 28807 leader_election.cc:302] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bf6e1a4b900440cbe4989be1460b2f8; no voters:
I20260805 23:15:42.123978 28807 leader_election.cc:288] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:42.124213 28813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:42.125056 28813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7bf6e1a4b900440cbe4989be1460b2f8, State: Running, Role: LEADER
I20260805 23:15:42.125463 28813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } }
I20260805 23:15:42.125566 28807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:42.127482 28814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } } }
I20260805 23:15:42.127652 28814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:42.127920 28815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf6e1a4b900440cbe4989be1460b2f8. Latest consensus state: current_term: 1 leader_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf6e1a4b900440cbe4989be1460b2f8" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36015 } } }
I20260805 23:15:42.128405 28815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8 [sys.catalog]: This master's current role is: LEADER
W20260805 23:15:42.130883 28829 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:42.130982 28829 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:42.131060 28828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:42.131832 28828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:42.137878 28828 catalog_manager.cc:1347] Generated new cluster ID: aa3efe93d09f4bf1b4b57af0dac39066
I20260805 23:15:42.137976 28828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:42.153174 28828 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:42.153574 28828 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:42.158522 28828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: Generated new TSK 0
I20260805 23:15:42.159479 28828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:42.176524 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--builtin_ntp_servers=127.23.88.84:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260805 23:15:42.304342 28833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:42.304687 28833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:42.304772 28833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:42.309273 28833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:42.309358 28833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:42.309422 28833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:42.314812 28833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:42.316125 28833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:42.317737 28833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:42.328588 28841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.328670 28838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.328589 28839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:42.329027 28833 server_base.cc:1034] running on GCE node
I20260805 23:15:42.329586 28833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:42.330345 28833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:42.333638 28833 hybrid_clock.cc:648] HybridClock initialized: now 1785971742333600 us; error 63 us; skew 500 ppm
I20260805 23:15:42.335728 28833 webserver.cc:460] Webserver started at http://127.23.88.65:33447/ using document root <none> and password file <none>
I20260805 23:15:42.336360 28833 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:42.336462 28833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:42.336720 28833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:42.338987 28833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "1427fe4af3c24aca83636341ae7a43d2"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "88440e57120337a28da1f5c8555919e5"
server_key_iv: "d5ba47538655dd6aebc653181b9a4c94"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.339689 28833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "1427fe4af3c24aca83636341ae7a43d2"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "88440e57120337a28da1f5c8555919e5"
server_key_iv: "d5ba47538655dd6aebc653181b9a4c94"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.343844 28833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:42.346621 28847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.348021 28833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260805 23:15:42.348170 28833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "1427fe4af3c24aca83636341ae7a43d2"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "88440e57120337a28da1f5c8555919e5"
server_key_iv: "d5ba47538655dd6aebc653181b9a4c94"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.348332 28833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:42.353300 28833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:42.353986 28833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:42.354189 28833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:42.354777 28833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:42.355794 28833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:42.355867 28833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.355955 28833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:42.356010 28833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.365398 28833 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:38493
I20260805 23:15:42.365424 28970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:38493 every 8 connection(s)
I20260805 23:15:42.366451 28833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:42.373561 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.368566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33897 (local address 127.23.88.126:36015)
0805 23:15:42.368734 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.368737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.369809 (+ 1072us) server_negotiation.cc:407] Connection header received
0805 23:15:42.371672 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.371676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.371729 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.371847 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.373041 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.373045 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.373049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.373063 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.373089 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.373103 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.373209 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.373357 (+ 148us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.373408 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:15:42.374480 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28833
I20260805 23:15:42.374620 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:42.374622 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.368976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36015 (local address 127.23.88.65:33897)
0805 23:15:42.369654 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.369697 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.371415 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.372046 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.372053 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.372146 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.372721 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.372723 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.372764 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.372833 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.373188 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.373197 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.373203 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.373404 (+ 201us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.373659 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":114,"threads_started":1}
I20260805 23:15:42.375003 23905 external_mini_cluster.cc:1428] Setting key a26e247d38291d88a78bdfe27f7333cf
I20260805 23:15:42.376008 28971 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36015
I20260805 23:15:42.376260 28971 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:42.376682 28971 heartbeater.cc:511] Master 127.23.88.126:36015 requested a full tablet report, sending...
I20260805 23:15:42.377439 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--builtin_ntp_servers=127.23.88.84:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260805 23:15:42.378285 28769 ts_manager.cc:194] Registered new tserver with Master: 1427fe4af3c24aca83636341ae7a43d2 (127.23.88.65:38493)
W20260805 23:15:42.493144 28974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:42.493444 28974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:42.493486 28974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:42.497761 28974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:42.497839 28974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:42.497900 28974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:42.502861 28974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:42.504160 28974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:42.505662 28974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:42.515954 28980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.515967 28979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.515954 28982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:42.516208 28974 server_base.cc:1034] running on GCE node
I20260805 23:15:42.516808 28974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:42.517611 28974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:42.519263 28974 hybrid_clock.cc:648] HybridClock initialized: now 1785971742519215 us; error 56 us; skew 500 ppm
I20260805 23:15:42.521652 28974 webserver.cc:460] Webserver started at http://127.23.88.66:46207/ using document root <none> and password file <none>
I20260805 23:15:42.522410 28974 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:42.522544 28974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:42.522858 28974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:42.525704 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "5a3f6bba4cda443e9751638a143e7fe6"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "852e5cf38a86e94e99c203abf5e40c3b"
server_key_iv: "d570ea60ee0cbff4a456f687af73e68a"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.526780 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "5a3f6bba4cda443e9751638a143e7fe6"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "852e5cf38a86e94e99c203abf5e40c3b"
server_key_iv: "d570ea60ee0cbff4a456f687af73e68a"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.531960 28974 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260805 23:15:42.536278 28989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.537880 28974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260805 23:15:42.538059 28974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "5a3f6bba4cda443e9751638a143e7fe6"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "852e5cf38a86e94e99c203abf5e40c3b"
server_key_iv: "d570ea60ee0cbff4a456f687af73e68a"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.538216 28974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:42.544135 28974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:42.544960 28974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:42.545202 28974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:42.545908 28974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:42.547156 28974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:42.547224 28974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.547317 28974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:42.547362 28974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.558578 29108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:39359 every 8 connection(s)
I20260805 23:15:42.559276 28974 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:39359
I20260805 23:15:42.560439 28974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:42.564951 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 28974
I20260805 23:15:42.565093 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:42.565488 23905 external_mini_cluster.cc:1428] Setting key af0476d9a0acc364b3e82981dfce2611
I20260805 23:15:42.568020 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.562583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:41725 (local address 127.23.88.126:36015)
0805 23:15:42.562746 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.562750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.564082 (+ 1332us) server_negotiation.cc:407] Connection header received
0805 23:15:42.565878 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.565886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.565949 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.566082 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.567497 (+ 1415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.567501 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.567503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.567517 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.567539 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.567550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.567688 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.567851 (+ 163us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.567896 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:15:42.568090 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--builtin_ntp_servers=127.23.88.84:33227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260805 23:15:42.569345 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.563157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36015 (local address 127.23.88.66:41725)
0805 23:15:42.563923 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.563966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.565561 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.566299 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.566309 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.566429 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.567175 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.567177 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.567222 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.567319 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.567680 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.567690 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.567698 (+ 8us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.567902 (+ 204us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.568185 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":133,"threads_started":1}
I20260805 23:15:42.570683 29109 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36015
I20260805 23:15:42.570952 29109 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:42.571424 29109 heartbeater.cc:511] Master 127.23.88.126:36015 requested a full tablet report, sending...
I20260805 23:15:42.572630 28769 ts_manager.cc:194] Registered new tserver with Master: 5a3f6bba4cda443e9751638a143e7fe6 (127.23.88.66:39359)
W20260805 23:15:42.692340 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:42.692698 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:42.692740 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:42.697265 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:42.697348 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:42.697419 29112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:42.702771 29112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36015
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:42.704115 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:42.705715 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:42.716666 29117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:42.716866 29112 server_base.cc:1034] running on GCE node
W20260805 23:15:42.716665 29120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:42.716742 29118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:42.717572 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:42.718324 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:42.719519 29112 hybrid_clock.cc:648] HybridClock initialized: now 1785971742719504 us; error 37 us; skew 500 ppm
I20260805 23:15:42.721614 29112 webserver.cc:460] Webserver started at http://127.23.88.67:45993/ using document root <none> and password file <none>
I20260805 23:15:42.722265 29112 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:42.722365 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:42.722618 29112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:42.724843 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "a834f328a4f349cea9b704554afe293b"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "c6fcf78fc87b5d3f877837b206d2abf6"
server_key_iv: "c56d8d6b6a947cd7d3e84e04483258a4"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.725571 29112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "a834f328a4f349cea9b704554afe293b"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "c6fcf78fc87b5d3f877837b206d2abf6"
server_key_iv: "c56d8d6b6a947cd7d3e84e04483258a4"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.729977 29112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:42.733011 29126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.734695 29112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:42.734853 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "a834f328a4f349cea9b704554afe293b"
format_stamp: "Formatted at 2026-08-05 23:15:42 on dist-test-slave-vkjp"
server_key: "c6fcf78fc87b5d3f877837b206d2abf6"
server_key_iv: "c56d8d6b6a947cd7d3e84e04483258a4"
server_key_version: "encryptionkey@0"
I20260805 23:15:42.735028 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:42.741786 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:42.742529 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:42.742693 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:42.743386 29112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:42.744482 29112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:42.744532 29112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.744599 29112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:42.744633 29112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:42.754441 29112 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35203
I20260805 23:15:42.754485 29248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35203 every 8 connection(s)
I20260805 23:15:42.755493 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:42.755897 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29112
I20260805 23:15:42.756027 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:42.756369 23905 external_mini_cluster.cc:1428] Setting key ecd6dda5e2517715ad521d982cf881dc
I20260805 23:15:42.762229 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.759220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:57801 (local address 127.23.88.126:36015)
0805 23:15:42.759356 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.759361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.759378 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:15:42.760530 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.760532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.760569 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.760642 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.761781 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.761787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.761794 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.761808 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.761838 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.761847 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.761943 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.762075 (+ 132us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.762115 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:42.763268 29251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.758184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36015 (local address 127.23.88.67:57801)
0805 23:15:42.759023 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.759109 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.760336 (+ 1227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.760822 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.760829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.760936 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.761503 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.761505 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.761546 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.761617 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.761949 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.761956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.761960 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.762098 (+ 138us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.762334 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":185,"threads_started":1}
I20260805 23:15:42.764531 29249 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36015
I20260805 23:15:42.764788 29249 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:42.765305 29249 heartbeater.cc:511] Master 127.23.88.126:36015 requested a full tablet report, sending...
I20260805 23:15:42.766498 28769 ts_manager.cc:194] Registered new tserver with Master: a834f328a4f349cea9b704554afe293b (127.23.88.67:35203)
I20260805 23:15:42.771142 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:15:42.775274 28810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.773330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58742 (local address 127.23.88.126:36015)
0805 23:15:42.773512 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.773518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.773648 (+ 130us) server_negotiation.cc:407] Connection header received
0805 23:15:42.773823 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.773830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.773870 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.773964 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.774795 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.774799 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.774802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.774815 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.774844 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.774852 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.774952 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.775032 (+ 80us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.775103 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260805 23:15:42.778467 28769 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:42.781852 28769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:42.795292 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.792461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:39359 (local address 127.0.0.1:44174)
0805 23:15:42.792815 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.792830 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.792911 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.794022 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.794025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.794041 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.794488 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.794490 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.794499 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.794519 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.794873 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.794887 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.794896 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.795019 (+ 123us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.795121 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":103,"threads_started":1}
I20260805 23:15:42.795507 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.791604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:38493 (local address 127.0.0.1:39398)
0805 23:15:42.792633 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.792689 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.792869 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.794168 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.794173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.794190 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.794780 (+ 590us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.794782 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.794792 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.794843 (+ 51us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.795258 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.795261 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.795264 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.795358 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.795395 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"thread_start_us":87,"threads_started":1}
I20260805 23:15:42.795856 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.791604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35203 (local address 127.0.0.1:58594)
0805 23:15:42.792686 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.792698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.792827 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.793625 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.793632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.793707 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.794339 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.794340 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.794380 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.794484 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.795168 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.795172 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.795176 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.795257 (+ 81us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.795293 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":161,"thread_start_us":110,"threads_started":1}
I20260805 23:15:42.795352 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.792686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.23.88.66:39359)
0805 23:15:42.793191 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.793205 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.793226 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:15:42.793272 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.793276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.793434 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.793550 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.794654 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.794659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.794673 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.794685 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.794732 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.794770 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.794883 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.795043 (+ 160us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.795171 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":98,"threads_started":1}
I20260805 23:15:42.796069 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.791748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.23.88.65:38493)
0805 23:15:42.792331 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.792341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.792812 (+ 471us) server_negotiation.cc:407] Connection header received
0805 23:15:42.793062 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.793068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.793278 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.793413 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.794989 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.794998 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.795008 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.795021 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.795066 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.795121 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.795217 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.795844 (+ 627us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.795921 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":164,"threads_started":1}
I20260805 23:15:42.796079 29263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.791851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.23.88.67:35203)
0805 23:15:42.793180 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.793193 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.793217 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:15:42.793267 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.793272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.793460 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.793598 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.794653 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.794659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.794673 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.794685 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.794732 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.794769 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.794883 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.795369 (+ 486us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.795458 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1220,"thread_start_us":99,"threads_started":1}
I20260805 23:15:42.798842 28899 tablet_service.cc:1467] Processing CreateTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 (DEFAULT_TABLE table=test-table [id=a8a2fdad855a4e068b9cb7dd0fef49f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:42.798956 29041 tablet_service.cc:1467] Processing CreateTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 (DEFAULT_TABLE table=test-table [id=a8a2fdad855a4e068b9cb7dd0fef49f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:42.798952 29180 tablet_service.cc:1467] Processing CreateTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 (DEFAULT_TABLE table=test-table [id=a8a2fdad855a4e068b9cb7dd0fef49f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:42.800061 29041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a61ffeecba543ca97542e6b3cf29801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:42.800061 29180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a61ffeecba543ca97542e6b3cf29801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:42.800062 28899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2a61ffeecba543ca97542e6b3cf29801. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:42.806883 29266 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Bootstrap starting.
I20260805 23:15:42.808123 29268 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Bootstrap starting.
I20260805 23:15:42.809520 29266 tablet_bootstrap.cc:654] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:42.810163 29268 tablet_bootstrap.cc:654] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:42.810415 29266 log.cc:824] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:42.810935 29268 log.cc:824] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:42.811159 29267 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Bootstrap starting.
I20260805 23:15:42.812700 29266 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: No bootstrap required, opened a new log
I20260805 23:15:42.812911 29266 ts_tablet_manager.cc:1397] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:15:42.812963 29268 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: No bootstrap required, opened a new log
I20260805 23:15:42.813118 29268 ts_tablet_manager.cc:1397] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260805 23:15:42.813185 29267 tablet_bootstrap.cc:654] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:42.813850 29267 log.cc:824] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:42.816099 29267 tablet_bootstrap.cc:492] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: No bootstrap required, opened a new log
I20260805 23:15:42.816309 29267 ts_tablet_manager.cc:1397] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:15:42.817819 29268 raft_consensus.cc:348] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.817819 29266 raft_consensus.cc:348] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.818117 29268 raft_consensus.cc:374] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:42.818116 29266 raft_consensus.cc:374] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:42.818177 29268 raft_consensus.cc:729] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3f6bba4cda443e9751638a143e7fe6, State: Initialized, Role: FOLLOWER
I20260805 23:15:42.818177 29266 raft_consensus.cc:729] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1427fe4af3c24aca83636341ae7a43d2, State: Initialized, Role: FOLLOWER
I20260805 23:15:42.818687 29266 consensus_queue.cc:260] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.818687 29268 consensus_queue.cc:260] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.819485 29109 heartbeater.cc:503] Master 127.23.88.126:36015 was elected leader, sending a full tablet report...
I20260805 23:15:42.819828 29268 ts_tablet_manager.cc:1428] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:15:42.820642 28971 heartbeater.cc:503] Master 127.23.88.126:36015 was elected leader, sending a full tablet report...
I20260805 23:15:42.821285 29267 raft_consensus.cc:348] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.821576 29267 raft_consensus.cc:374] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:42.821640 29267 raft_consensus.cc:729] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a834f328a4f349cea9b704554afe293b, State: Initialized, Role: FOLLOWER
I20260805 23:15:42.822021 29266 ts_tablet_manager.cc:1428] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260805 23:15:42.822201 29267 consensus_queue.cc:260] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.822963 29249 heartbeater.cc:503] Master 127.23.88.126:36015 was elected leader, sending a full tablet report...
I20260805 23:15:42.823318 29267 ts_tablet_manager.cc:1428] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
W20260805 23:15:42.867707 28972 tablet.cc:2369] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:42.872460 29272 raft_consensus.cc:482] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:42.872745 29272 raft_consensus.cc:504] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.874097 29272 leader_election.cc:288] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a834f328a4f349cea9b704554afe293b (127.23.88.67:35203), 5a3f6bba4cda443e9751638a143e7fe6 (127.23.88.66:39359)
I20260805 23:15:42.876549 29263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.874624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33797 (local address 127.23.88.67:35203)
0805 23:15:42.874801 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.874806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.874824 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:42.875002 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.875005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.875055 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.875169 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.876108 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.876111 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.876116 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.876130 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.876155 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.876166 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.876271 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.876366 (+ 95us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.876409 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:15:42.876557 29265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.875034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33923 (local address 127.23.88.66:39359)
0805 23:15:42.875199 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.875202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.875305 (+ 103us) server_negotiation.cc:407] Connection header received
0805 23:15:42.875412 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.875414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.875445 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.875513 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.876154 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.876157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.876161 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.876173 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.876196 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.876208 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.876295 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.876368 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.876409 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:15:42.876575 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.874559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35203 (local address 127.23.88.65:33797)
0805 23:15:42.874727 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.874745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.874890 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.875287 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.875291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.875308 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.875862 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.875864 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.875876 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.875941 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.876276 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.876280 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.876283 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.876366 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.876429 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":19}
I20260805 23:15:42.876590 29275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.874844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:39359 (local address 127.23.88.65:33923)
0805 23:15:42.875243 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:42.875256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:42.875329 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:42.875563 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:42.875566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:42.875580 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:42.875982 (+ 402us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:15:42.875986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:42.875998 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:15:42.876024 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:42.876285 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:42.876287 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:42.876290 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:15:42.876365 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:15:42.876419 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":126,"threads_started":1}
I20260805 23:15:42.877249 29201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a61ffeecba543ca97542e6b3cf29801" candidate_uuid: "1427fe4af3c24aca83636341ae7a43d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a834f328a4f349cea9b704554afe293b" is_pre_election: true
I20260805 23:15:42.877439 29061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a61ffeecba543ca97542e6b3cf29801" candidate_uuid: "1427fe4af3c24aca83636341ae7a43d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f6bba4cda443e9751638a143e7fe6" is_pre_election: true
I20260805 23:15:42.877606 29201 raft_consensus.cc:2454] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1427fe4af3c24aca83636341ae7a43d2 in term 0.
I20260805 23:15:42.877750 29061 raft_consensus.cc:2454] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1427fe4af3c24aca83636341ae7a43d2 in term 0.
I20260805 23:15:42.878131 28848 leader_election.cc:302] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1427fe4af3c24aca83636341ae7a43d2, a834f328a4f349cea9b704554afe293b; no voters:
I20260805 23:15:42.878412 29272 raft_consensus.cc:2781] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:42.878507 29272 raft_consensus.cc:482] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:42.878553 29272 raft_consensus.cc:3037] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:42.941836 29272 raft_consensus.cc:504] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.942536 29272 leader_election.cc:288] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [CANDIDATE]: Term 1 election: Requested vote from peers a834f328a4f349cea9b704554afe293b (127.23.88.67:35203), 5a3f6bba4cda443e9751638a143e7fe6 (127.23.88.66:39359)
I20260805 23:15:42.943152 29201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a61ffeecba543ca97542e6b3cf29801" candidate_uuid: "1427fe4af3c24aca83636341ae7a43d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a834f328a4f349cea9b704554afe293b"
I20260805 23:15:42.943256 29061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2a61ffeecba543ca97542e6b3cf29801" candidate_uuid: "1427fe4af3c24aca83636341ae7a43d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3f6bba4cda443e9751638a143e7fe6"
I20260805 23:15:42.943346 29201 raft_consensus.cc:3037] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:42.943419 29061 raft_consensus.cc:3037] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:42.945001 29201 raft_consensus.cc:2454] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1427fe4af3c24aca83636341ae7a43d2 in term 1.
I20260805 23:15:42.945001 29061 raft_consensus.cc:2454] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1427fe4af3c24aca83636341ae7a43d2 in term 1.
I20260805 23:15:42.945456 28848 leader_election.cc:302] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1427fe4af3c24aca83636341ae7a43d2, 5a3f6bba4cda443e9751638a143e7fe6; no voters:
I20260805 23:15:42.945720 29272 raft_consensus.cc:2781] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:42.946081 29272 raft_consensus.cc:686] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [term 1 LEADER]: Becoming Leader. State: Replica: 1427fe4af3c24aca83636341ae7a43d2, State: Running, Role: LEADER
I20260805 23:15:42.946497 29272 consensus_queue.cc:237] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } }
I20260805 23:15:42.949990 28767 catalog_manager.cc:5495] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1427fe4af3c24aca83636341ae7a43d2 (127.23.88.65). New cstate: current_term: 1 leader_uuid: "1427fe4af3c24aca83636341ae7a43d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1427fe4af3c24aca83636341ae7a43d2" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 38493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:42.991019 29262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:42.989671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.23.88.65:38493)
0805 23:15:42.989812 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:42.989815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:42.989846 (+ 31us) server_negotiation.cc:407] Connection header received
0805 23:15:42.989929 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:42.989931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:42.989998 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:42.990066 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:42.990694 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:42.990697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:42.990699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:42.990710 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:42.990729 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:42.990737 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:42.990807 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:42.990868 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:15:42.990905 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:15:42.997624 29201 raft_consensus.cc:1261] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 1 FOLLOWER]: Refusing update from remote peer 1427fe4af3c24aca83636341ae7a43d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:42.997624 29061 raft_consensus.cc:1261] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 1 FOLLOWER]: Refusing update from remote peer 1427fe4af3c24aca83636341ae7a43d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:42.998525 29272 consensus_queue.cc:1035] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3f6bba4cda443e9751638a143e7fe6" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 39359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:42.998776 29276 consensus_queue.cc:1035] T 2a61ffeecba543ca97542e6b3cf29801 P 1427fe4af3c24aca83636341ae7a43d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a834f328a4f349cea9b704554afe293b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:43.006791 29280 mvcc.cc:205] Tried to move back new op lower bound from 7315340259309699072 to 7315340259112034304. Current Snapshot: MvccSnapshot[applied={T|T < 7315340259309699072}]
I20260805 23:15:43.007982 29279 mvcc.cc:205] Tried to move back new op lower bound from 7315340259309699072 to 7315340259112034304. Current Snapshot: MvccSnapshot[applied={T|T < 7315340259309699072}]
I20260805 23:15:43.018472 28767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:43.018711 28767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:43.021807 28767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7bf6e1a4b900440cbe4989be1460b2f8: Sending DeleteTablet for 3 replicas of tablet 2a61ffeecba543ca97542e6b3cf29801
I20260805 23:15:43.022913 28899 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:43 UTC) from {username='slave'} at 127.0.0.1:39398
I20260805 23:15:43.023046 29180 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:43 UTC) from {username='slave'} at 127.0.0.1:58594
I20260805 23:15:43.023303 29041 tablet_service.cc:1514] Processing DeleteTablet for tablet 2a61ffeecba543ca97542e6b3cf29801 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:43 UTC) from {username='slave'} at 127.0.0.1:44174
I20260805 23:15:43.023808 29289 tablet_replica.cc:330] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: stopping tablet replica
I20260805 23:15:43.024133 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28833
I20260805 23:15:43.025727 29291 tablet_replica.cc:330] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: stopping tablet replica
I20260805 23:15:43.026248 29289 raft_consensus.cc:2229] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:43.026430 29291 raft_consensus.cc:2229] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:43.026674 29289 raft_consensus.cc:2258] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:43.026805 29291 raft_consensus.cc:2258] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:43.028580 29289 ts_tablet_manager.cc:1905] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:43.031447 29291 ts_tablet_manager.cc:1905] T 2a61ffeecba543ca97542e6b3cf29801 P 5a3f6bba4cda443e9751638a143e7fe6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:15:43.033612 28753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:38493 (error 108)
I20260805 23:15:43.034629 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28974
I20260805 23:15:43.036939 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.034792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:38493 (local address 127.0.0.1:39414)
0805 23:15:43.036712 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:43.036810 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:38493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1800}
W20260805 23:15:43.038231 28753 catalog_manager.cc:4584] TS 1427fe4af3c24aca83636341ae7a43d2 (127.23.88.65:38493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2a61ffeecba543ca97542e6b3cf29801: Network error: Client connection negotiation failed: client connection to 127.23.88.65:38493: connect: Connection refused (error 111)
I20260805 23:15:43.042726 29289 ts_tablet_manager.cc:1918] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:43.042893 29289 log.cc:1196] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/2a61ffeecba543ca97542e6b3cf29801
I20260805 23:15:43.043311 29289 ts_tablet_manager.cc:1939] T 2a61ffeecba543ca97542e6b3cf29801 P a834f328a4f349cea9b704554afe293b: Deleting consensus metadata
I20260805 23:15:43.047881 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29112
I20260805 23:15:43.051683 29264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.050352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:39359 (local address 127.0.0.1:44178)
0805 23:15:43.051528 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:43.051588 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:39359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1071}
I20260805 23:15:43.055735 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 28737
2026-08-05T23:15:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1198 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260805 23:15:43.078926 23905 mini_oidc.cc:152] Starting JWKS server
I20260805 23:15:43.080829 23905 webserver.cc:460] Webserver started at https://127.0.0.1:44285/ using document root <none> and password file <none>
I20260805 23:15:43.081008 23905 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:15:43.081274 23905 webserver.cc:460] Webserver started at http://127.0.0.1:38045/ using document root <none> and password file <none>
2026-08-05T23:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:43Z Disabled control of system clock
I20260805 23:15:43.091902 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:40549
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:36135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:40549
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44285/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/testchainca.cert with env {}
W20260805 23:15:43.208165 29298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:43.208472 29298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:43.208519 29298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:43.212711 29298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:43.212814 29298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:43.212834 29298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:43.212854 29298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:43.212873 29298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:15:43.212893 29298 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:15:43.212910 29298 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44285/jwks.json
I20260805 23:15:43.218652 29298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:40549
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:40549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44285/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:43.219887 29298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:43.220898 29298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:43.229796 29306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:43.229741 29308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:43.229871 29298 server_base.cc:1034] running on GCE node
W20260805 23:15:43.229753 29305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:43.230786 29298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:43.232095 29298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:43.233315 29298 hybrid_clock.cc:648] HybridClock initialized: now 1785971743233285 us; error 47 us; skew 500 ppm
I20260805 23:15:43.239485 29298 webserver.cc:460] Webserver started at http://127.23.88.126:41869/ using document root <none> and password file <none>
I20260805 23:15:43.240155 29298 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:43.240260 29298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:43.240540 29298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:43.242780 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "98ff5dde05a4427eafdedf8636edce38"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "44fcefb62ce131ef58b4007a67385351"
server_key_iv: "c988bcb367808dfe0f631c649ba43a27"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.243506 29298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "98ff5dde05a4427eafdedf8636edce38"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "44fcefb62ce131ef58b4007a67385351"
server_key_iv: "c988bcb367808dfe0f631c649ba43a27"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.247661 29298 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:15:43.250553 29314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:43.252286 29298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:43.252444 29298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "98ff5dde05a4427eafdedf8636edce38"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "44fcefb62ce131ef58b4007a67385351"
server_key_iv: "c988bcb367808dfe0f631c649ba43a27"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.252593 29298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:43.282409 29298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:43.283304 29298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:43.283532 29298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:43.290856 29298 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:40549
I20260805 23:15:43.291044 29370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:40549 every 8 connection(s)
I20260805 23:15:43.291970 29298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:43.295131 29371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:43.298959 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29298
I20260805 23:15:43.299125 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:43.299466 23905 external_mini_cluster.cc:1428] Setting key 6ed6c59c06cb1bc5729e2a504d12797b
I20260805 23:15:43.301530 29371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38: Bootstrap starting.
I20260805 23:15:43.304008 29371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:43.304790 29371 log.cc:824] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:43.306727 29371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38: No bootstrap required, opened a new log
I20260805 23:15:43.309782 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.300972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.23.88.126:40549)
0805 23:15:43.301746 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:43.301766 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:43.301809 (+ 43us) server_negotiation.cc:407] Connection header received
0805 23:15:43.302687 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:43.302706 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:43.303194 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:43.303526 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:43.303991 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.305059 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.305973 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.306447 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.306620 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.307606 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:43.307612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:43.307624 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:43.308044 (+ 420us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:43.308104 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:43.308126 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:43.308252 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:43.308461 (+ 209us) server_negotiation.cc:299] Negotiation successful
0805 23:15:43.308733 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":147,"threads_started":1}
I20260805 23:15:43.310199 29371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } }
I20260805 23:15:43.310456 29371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:43.310520 29371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98ff5dde05a4427eafdedf8636edce38, State: Initialized, Role: FOLLOWER
I20260805 23:15:43.310947 29371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } }
I20260805 23:15:43.311149 29371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:43.311234 29371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:43.311362 29371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:43.313047 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:43.351563 29371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } }
I20260805 23:15:43.352100 29371 leader_election.cc:302] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98ff5dde05a4427eafdedf8636edce38; no voters:
I20260805 23:15:43.352596 29371 leader_election.cc:288] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:43.352813 29376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:43.353190 29376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [term 1 LEADER]: Becoming Leader. State: Replica: 98ff5dde05a4427eafdedf8636edce38, State: Running, Role: LEADER
I20260805 23:15:43.353574 29376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } }
I20260805 23:15:43.354085 29371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:43.355443 29377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98ff5dde05a4427eafdedf8636edce38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } } }
I20260805 23:15:43.355559 29378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98ff5dde05a4427eafdedf8636edce38. Latest consensus state: current_term: 1 leader_uuid: "98ff5dde05a4427eafdedf8636edce38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98ff5dde05a4427eafdedf8636edce38" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40549 } } }
I20260805 23:15:43.355610 29377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:43.355635 29378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:43.356056 29385 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:43.358889 29385 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:43.365317 29385 catalog_manager.cc:1347] Generated new cluster ID: d99301d746aa40bab3502c9085347037
I20260805 23:15:43.365430 29385 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:43.386065 29385 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:43.387176 29385 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:43.397028 29385 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98ff5dde05a4427eafdedf8636edce38: Generated new TSK 0
I20260805 23:15:43.397816 29385 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 23:15:43.431181 29401 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260805 23:15:43.431376 29401 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260805 23:15:43.431960 29374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.413628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59764 (local address 127.23.88.126:40549)
0805 23:15:43.413793 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:43.413800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:43.414016 (+ 216us) server_negotiation.cc:407] Connection header received
0805 23:15:43.414256 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:43.414259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:43.414316 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:43.414449 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:15:43.414796 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.415490 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.416717 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.416963 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.417082 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.417543 (+ 461us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:15:43.431576 (+ 14033us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 23:15:43.431841 (+ 265us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260805 23:15:43.432070 29374 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260805 23:15:43.434056 29400 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:43.413528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:40549 (local address 127.0.0.1:59764)
0805 23:15:43.413911 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:43.413927 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:43.414079 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:43.414489 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:43.414493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:43.414502 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:15:43.414643 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:43.414650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.415613 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.415616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:43.416580 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:43.416590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.417104 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.417108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:43.417204 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.417277 (+ 73us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:15:43.431848 (+ 14571us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0805 23:15:43.432488 (+ 640us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.88.126:40549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":124,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.88.126:40549: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44285/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260805 23:15:43.434876 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29298
2026-08-05T23:15:43Z chronyd exiting
I20260805 23:15:43.473424 23905 test_util.cc:175] -----------------------------------------------
I20260805 23:15:43.473519 23905 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785971715702001-23905-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (396 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260805 23:15:43.474655 23905 mini_oidc.cc:152] Starting JWKS server
I20260805 23:15:43.475648 23905 webserver.cc:460] Webserver started at https://127.0.0.1:36787/ using document root <none> and password file <none>
I20260805 23:15:43.475773 23905 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:15:43.476065 23905 webserver.cc:460] Webserver started at http://127.0.0.1:43421/ using document root <none> and password file <none>
2026-08-05T23:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:43Z Disabled control of system clock
I20260805 23:15:43.486100 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:40045
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:40965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:40045
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36787/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/testchainca.cert with env {}
W20260805 23:15:43.606030 29407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:43.606333 29407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:43.606375 29407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:43.610621 29407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:43.610702 29407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:43.610718 29407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:43.610734 29407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:43.610750 29407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:15:43.610766 29407 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:15:43.610780 29407 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36787/jwks.json
I20260805 23:15:43.616566 29407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:40965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:40045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:40045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36787/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:43.617636 29407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:43.618600 29407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:43.627449 29415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:43.627457 29413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:43.627457 29412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:43.628003 29407 server_base.cc:1034] running on GCE node
I20260805 23:15:43.628561 29407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:43.629751 29407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:43.630940 29407 hybrid_clock.cc:648] HybridClock initialized: now 1785971743630921 us; error 34 us; skew 500 ppm
I20260805 23:15:43.633014 29407 webserver.cc:460] Webserver started at http://127.23.88.126:36531/ using document root <none> and password file <none>
I20260805 23:15:43.633641 29407 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:43.633709 29407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:43.633944 29407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:43.636186 29407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "d64fa6c3d9e64d6d94e6df8565187448"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "1de66535d774951aef8dcd92d568b754"
server_key_iv: "7e448df9b77134afa08853a82a98536b"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.636763 29407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "d64fa6c3d9e64d6d94e6df8565187448"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "1de66535d774951aef8dcd92d568b754"
server_key_iv: "7e448df9b77134afa08853a82a98536b"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.640733 29407 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:43.643548 29421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:43.644932 29407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:43.645128 29407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "d64fa6c3d9e64d6d94e6df8565187448"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "1de66535d774951aef8dcd92d568b754"
server_key_iv: "7e448df9b77134afa08853a82a98536b"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.645298 29407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:43.658948 29407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:43.659796 29407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:43.660036 29407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:43.667589 29407 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:40045
I20260805 23:15:43.667641 29477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:40045 every 8 connection(s)
I20260805 23:15:43.668628 29407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:43.672122 29478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:43.672981 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29407
I20260805 23:15:43.673103 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:43.673421 23905 external_mini_cluster.cc:1428] Setting key 37cc4f1ffd5ebf30c5a7e7b8ff429d7e
I20260805 23:15:43.678407 29478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448: Bootstrap starting.
I20260805 23:15:43.680765 29478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:43.681535 29478 log.cc:824] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:43.683048 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.674803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35170 (local address 127.23.88.126:40045)
0805 23:15:43.675566 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:43.675583 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:43.675619 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:15:43.676376 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:43.676396 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:43.676843 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:43.677182 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:43.677626 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.678639 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.679586 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.680054 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.680238 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.681143 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:43.681148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:43.681160 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:43.681564 (+ 404us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:43.681619 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:43.681639 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:43.681766 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:43.681932 (+ 166us) server_negotiation.cc:299] Negotiation successful
0805 23:15:43.682222 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":169,"threads_started":1}
I20260805 23:15:43.683914 29478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448: No bootstrap required, opened a new log
I20260805 23:15:43.685961 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:43.688050 29478 raft_consensus.cc:348] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } }
I20260805 23:15:43.688277 29478 raft_consensus.cc:374] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:43.688361 29478 raft_consensus.cc:729] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d64fa6c3d9e64d6d94e6df8565187448, State: Initialized, Role: FOLLOWER
I20260805 23:15:43.688804 29478 consensus_queue.cc:260] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } }
I20260805 23:15:43.688956 29478 raft_consensus.cc:388] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:43.689049 29478 raft_consensus.cc:482] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:43.689142 29478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:43.690117 29478 raft_consensus.cc:504] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } }
I20260805 23:15:43.690474 29478 leader_election.cc:302] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d64fa6c3d9e64d6d94e6df8565187448; no voters:
I20260805 23:15:43.690821 29478 leader_election.cc:288] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:43.691017 29484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:43.691290 29484 raft_consensus.cc:686] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [term 1 LEADER]: Becoming Leader. State: Replica: d64fa6c3d9e64d6d94e6df8565187448, State: Running, Role: LEADER
I20260805 23:15:43.691627 29484 consensus_queue.cc:237] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } }
I20260805 23:15:43.692207 29478 sys_catalog.cc:564] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:43.693228 29485 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d64fa6c3d9e64d6d94e6df8565187448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } } }
I20260805 23:15:43.693367 29485 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:43.693804 29493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:43.693771 29486 sys_catalog.cc:455] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d64fa6c3d9e64d6d94e6df8565187448. Latest consensus state: current_term: 1 leader_uuid: "d64fa6c3d9e64d6d94e6df8565187448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d64fa6c3d9e64d6d94e6df8565187448" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 40045 } } }
I20260805 23:15:43.693918 29486 sys_catalog.cc:458] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:43.697113 29493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:43.703202 29493 catalog_manager.cc:1347] Generated new cluster ID: b2d3720c263f4b40a9ea376b12ed602d
I20260805 23:15:43.703282 29493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:43.733659 29493 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:43.734719 29493 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:43.744915 29493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d64fa6c3d9e64d6d94e6df8565187448: Generated new TSK 0
I20260805 23:15:43.745694 29493 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260805 23:15:43.783548 29510 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260805 23:15:43.783689 29510 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260805 23:15:43.784384 29481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:43.765046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35186 (local address 127.23.88.126:40045)
0805 23:15:43.765183 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:43.765188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:43.765392 (+ 204us) server_negotiation.cc:407] Connection header received
0805 23:15:43.765580 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:43.765583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:43.765648 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:43.765775 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:15:43.766162 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.767026 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.768437 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.768802 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.768877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.769361 (+ 484us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:15:43.783935 (+ 14574us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 23:15:43.784229 (+ 294us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":54}
W20260805 23:15:43.784459 29481 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35186: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260805 23:15:43.784552 29509 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:43.764932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:40045 (local address 127.0.0.1:35186)
0805 23:15:43.765287 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:43.765303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:43.765425 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:43.765898 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:43.765902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:43.765911 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:15:43.766028 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:43.766035 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.767245 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.767249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:43.768275 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:43.768287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:43.769023 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:43.769027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:43.769121 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:43.769151 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:15:43.784179 (+ 15028us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0805 23:15:43.784352 (+ 173us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.88.126:40045: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36787/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":108,"threads_started":1}
I20260805 23:15:43.785254 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29407
2026-08-05T23:15:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (352 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260805 23:15:43.826689 23905 mini_oidc.cc:152] Starting JWKS server
I20260805 23:15:43.827647 23905 webserver.cc:460] Webserver started at https://127.0.0.1:36357/ using document root <none> and password file <none>
I20260805 23:15:43.827733 23905 mini_oidc.cc:189] Starting OIDC Discovery server
I20260805 23:15:43.827957 23905 webserver.cc:460] Webserver started at http://127.0.0.1:43019/ using document root <none> and password file <none>
2026-08-05T23:15:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:43Z Disabled control of system clock
I20260805 23:15:43.837612 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44445
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:46055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:44445
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36357/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 23:15:43.953907 29516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:43.954206 29516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:43.954249 29516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:43.958499 29516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:43.958588 29516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:43.958604 29516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:43.958622 29516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:43.958642 29516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:15:43.958658 29516 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:15:43.958673 29516 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36357/jwks.json
I20260805 23:15:43.963825 29516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:44445
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36357/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:43.964977 29516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:43.966428 29516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:43.975344 29522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:43.975329 29524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:43.975329 29521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:43.975988 29516 server_base.cc:1034] running on GCE node
I20260805 23:15:43.976650 29516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:43.977954 29516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:43.979198 29516 hybrid_clock.cc:648] HybridClock initialized: now 1785971743979168 us; error 36 us; skew 500 ppm
I20260805 23:15:43.981336 29516 webserver.cc:460] Webserver started at http://127.23.88.126:46767/ using document root <none> and password file <none>
I20260805 23:15:43.981982 29516 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:43.982085 29516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:43.982336 29516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:43.984702 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "3250bb41f0ab45fd84b32eeb87818848"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "4028579fc76f030eab4d0026202d69e1"
server_key_iv: "8e0c1da05b6a9ee52eb818f81b4a9444"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.985298 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "3250bb41f0ab45fd84b32eeb87818848"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "4028579fc76f030eab4d0026202d69e1"
server_key_iv: "8e0c1da05b6a9ee52eb818f81b4a9444"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.989948 29516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:15:43.992875 29530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:43.994343 29516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:43.994524 29516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3250bb41f0ab45fd84b32eeb87818848"
format_stamp: "Formatted at 2026-08-05 23:15:43 on dist-test-slave-vkjp"
server_key: "4028579fc76f030eab4d0026202d69e1"
server_key_iv: "8e0c1da05b6a9ee52eb818f81b4a9444"
server_key_version: "encryptionkey@0"
I20260805 23:15:43.994663 29516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:44.008833 29516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:44.009622 29516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:44.009845 29516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:44.017202 29516 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:44445
I20260805 23:15:44.017231 29584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:44445 every 8 connection(s)
I20260805 23:15:44.018211 29516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:44.021566 29585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:44.024497 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29516
I20260805 23:15:44.024608 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:44.024910 23905 external_mini_cluster.cc:1428] Setting key 6a027db5ed45292481672a0c0a0743cb
I20260805 23:15:44.027984 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848: Bootstrap starting.
I20260805 23:15:44.030462 29585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:44.031343 29585 log.cc:824] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:44.033392 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848: No bootstrap required, opened a new log
I20260805 23:15:44.034644 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:44.026374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.23.88.126:44445)
0805 23:15:44.027070 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:44.027154 (+ 84us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:44.027195 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:15:44.027890 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:44.027908 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:44.028418 (+ 510us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:44.028769 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:44.029237 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.030202 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.031187 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.031730 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.031902 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.032788 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:44.032793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:44.032807 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:44.033195 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:44.033260 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:44.033279 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:44.033390 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:44.033509 (+ 119us) server_negotiation.cc:299] Negotiation successful
0805 23:15:44.033776 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":139,"threads_started":1}
I20260805 23:15:44.037742 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:44.037849 29585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } }
I20260805 23:15:44.038129 29585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:44.038208 29585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3250bb41f0ab45fd84b32eeb87818848, State: Initialized, Role: FOLLOWER
I20260805 23:15:44.038686 29585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } }
I20260805 23:15:44.038872 29585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:44.038964 29585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:44.039068 29585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:44.040130 29585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } }
I20260805 23:15:44.040436 29585 leader_election.cc:302] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3250bb41f0ab45fd84b32eeb87818848; no voters:
I20260805 23:15:44.040707 29585 leader_election.cc:288] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:44.040848 29590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:44.041123 29590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [term 1 LEADER]: Becoming Leader. State: Replica: 3250bb41f0ab45fd84b32eeb87818848, State: Running, Role: LEADER
I20260805 23:15:44.041435 29590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } }
I20260805 23:15:44.042035 29585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:44.043303 29591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3250bb41f0ab45fd84b32eeb87818848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } } }
I20260805 23:15:44.043308 29592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3250bb41f0ab45fd84b32eeb87818848. Latest consensus state: current_term: 1 leader_uuid: "3250bb41f0ab45fd84b32eeb87818848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3250bb41f0ab45fd84b32eeb87818848" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44445 } } }
I20260805 23:15:44.043509 29592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:44.043478 29591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:44.043924 29599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:44.046801 29599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:44.052964 29599 catalog_manager.cc:1347] Generated new cluster ID: 741160868b564d0d97da0ab7c51f1686
I20260805 23:15:44.053073 29599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:44.069339 29599 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:44.070477 29599 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:44.076368 29599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3250bb41f0ab45fd84b32eeb87818848: Generated new TSK 0
I20260805 23:15:44.077090 29599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:44.092873 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:44.087814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38218 (local address 127.23.88.126:44445)
0805 23:15:44.087971 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:44.087978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:44.088159 (+ 181us) server_negotiation.cc:407] Connection header received
0805 23:15:44.088349 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:44.088351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:44.088401 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:44.088521 (+ 120us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:15:44.088822 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.089347 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.090594 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.090813 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.090888 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.091249 (+ 361us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:15:44.092467 (+ 1218us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36357/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 23:15:44.092713 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38218: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36357/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":63}
W20260805 23:15:44.092937 29615 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:44.087713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44445 (local address 127.0.0.1:38218)
0805 23:15:44.088060 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:44.088076 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:44.088200 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:44.088557 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:44.088560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:44.088568 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:15:44.088692 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:44.088699 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.089472 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.089476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:44.090475 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:44.090485 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.090948 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.090952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:44.091039 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.091068 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:15:44.092628 (+ 1560us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36357/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0805 23:15:44.092767 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.88.126:44445: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36357/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":123,"threads_started":1}
W20260805 23:15:44.092953 29588 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38218: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36357/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260805 23:15:44.093364 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29516
E20260805 23:15:44.104760 29616 webserver.cc:533] Webserver: sslize failed:
E20260805 23:15:44.104843 29616 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-05T23:15:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (306 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-05T23:15:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:44Z Disabled control of system clock
I20260805 23:15:44.142601 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42527
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:35935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42527
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/testchainca.cert with env {}
W20260805 23:15:44.258256 29620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:44.258549 29620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:44.258594 29620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:44.262779 29620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:44.262856 29620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:44.262872 29620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:44.262892 29620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:44.262907 29620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260805 23:15:44.262964 29620 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260805 23:15:44.262991 29620 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260805 23:15:44.268894 29620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42527
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:44.270071 29620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:44.271167 29620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:44.279973 29620 server_base.cc:1034] running on GCE node
W20260805 23:15:44.279889 29626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:44.279938 29625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:44.279889 29628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:44.280880 29620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:44.282076 29620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:44.283303 29620 hybrid_clock.cc:648] HybridClock initialized: now 1785971744283285 us; error 35 us; skew 500 ppm
I20260805 23:15:44.285624 29620 webserver.cc:460] Webserver started at http://127.23.88.126:46285/ using document root <none> and password file <none>
I20260805 23:15:44.286226 29620 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:44.286322 29620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:44.286547 29620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:44.288766 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "004db9623615474c88eafd586c70f3f7"
format_stamp: "Formatted at 2026-08-05 23:15:44 on dist-test-slave-vkjp"
server_key: "e9918ceb37e41f4f1d47f0a79bbac4a5"
server_key_iv: "b3341a4c2e8147bf4bbc6ec7b487a256"
server_key_version: "encryptionkey@0"
I20260805 23:15:44.289363 29620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "004db9623615474c88eafd586c70f3f7"
format_stamp: "Formatted at 2026-08-05 23:15:44 on dist-test-slave-vkjp"
server_key: "e9918ceb37e41f4f1d47f0a79bbac4a5"
server_key_iv: "b3341a4c2e8147bf4bbc6ec7b487a256"
server_key_version: "encryptionkey@0"
I20260805 23:15:44.293238 29620 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:44.295835 29634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:44.297252 29620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:44.297464 29620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "004db9623615474c88eafd586c70f3f7"
format_stamp: "Formatted at 2026-08-05 23:15:44 on dist-test-slave-vkjp"
server_key: "e9918ceb37e41f4f1d47f0a79bbac4a5"
server_key_iv: "b3341a4c2e8147bf4bbc6ec7b487a256"
server_key_version: "encryptionkey@0"
I20260805 23:15:44.297605 29620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:44.319823 29620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:44.320578 29620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:44.320788 29620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:44.328168 29620 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42527
I20260805 23:15:44.328191 29688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42527 every 8 connection(s)
I20260805 23:15:44.329161 29620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:44.332286 29689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:44.339069 29689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7: Bootstrap starting.
I20260805 23:15:44.339152 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29620
I20260805 23:15:44.339270 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:44.339551 23905 external_mini_cluster.cc:1428] Setting key c3bba6c11dce3565376dda8db190ee8f
I20260805 23:15:44.341974 29689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:44.342751 29689 log.cc:824] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:44.345140 29689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7: No bootstrap required, opened a new log
I20260805 23:15:44.349695 29689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } }
I20260805 23:15:44.349982 29689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:44.350035 29689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 004db9623615474c88eafd586c70f3f7, State: Initialized, Role: FOLLOWER
I20260805 23:15:44.350113 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:44.340983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.23.88.126:42527)
0805 23:15:44.341689 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:44.341704 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:44.341739 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:15:44.342485 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:44.342506 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:44.342976 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:44.343332 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:44.343749 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.345197 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.346105 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.346800 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.346981 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.347822 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:44.347829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:44.347848 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:15:44.348408 (+ 560us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:44.348498 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:15:44.348528 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:44.348656 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:44.348802 (+ 146us) server_negotiation.cc:299] Negotiation successful
0805 23:15:44.349113 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":146,"threads_started":1}
I20260805 23:15:44.350525 29689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } }
I20260805 23:15:44.350687 29689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:44.350752 29689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:44.350837 29689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:44.351876 29689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } }
I20260805 23:15:44.352252 29689 leader_election.cc:302] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 004db9623615474c88eafd586c70f3f7; no voters:
I20260805 23:15:44.352628 29689 leader_election.cc:288] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:44.352710 29695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:44.352931 29695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [term 1 LEADER]: Becoming Leader. State: Replica: 004db9623615474c88eafd586c70f3f7, State: Running, Role: LEADER
I20260805 23:15:44.353117 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:15:44.353229 29695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } }
I20260805 23:15:44.353963 29689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:44.354458 29696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 004db9623615474c88eafd586c70f3f7. Latest consensus state: current_term: 1 leader_uuid: "004db9623615474c88eafd586c70f3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } } }
I20260805 23:15:44.354578 29696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:44.355507 29697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "004db9623615474c88eafd586c70f3f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "004db9623615474c88eafd586c70f3f7" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42527 } } }
I20260805 23:15:44.355611 29697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:44.356324 29700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:44.359283 29700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:44.364823 29700 catalog_manager.cc:1347] Generated new cluster ID: 432c9da410df409dbef46b857360b60e
I20260805 23:15:44.364912 29700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:44.395406 29700 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:44.396611 29700 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:44.406118 29700 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 004db9623615474c88eafd586c70f3f7: Generated new TSK 0
I20260805 23:15:44.406862 29700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:44.485100 29693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:44.425018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38324 (local address 127.23.88.126:42527)
0805 23:15:44.425168 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:44.425174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:44.425314 (+ 140us) server_negotiation.cc:407] Connection header received
0805 23:15:44.425469 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:44.425472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:44.425531 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:44.425650 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0805 23:15:44.425882 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.426692 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.427607 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.427955 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.428054 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.428342 (+ 288us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0805 23:15:44.484552 (+ 56210us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 23:15:44.484902 (+ 350us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38324: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":57}
W20260805 23:15:44.485215 29693 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38324: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260805 23:15:44.485262 29720 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:44.424912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42527 (local address 127.0.0.1:38324)
0805 23:15:44.425221 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:44.425234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:44.425343 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:44.425653 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:44.425655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:44.425661 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0805 23:15:44.425776 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:44.425780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.426828 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.426830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:44.427492 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:44.427500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:44.428081 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:44.428084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:44.428145 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:44.428168 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0805 23:15:44.484836 (+ 56668us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0805 23:15:44.485031 (+ 195us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.88.126:42527: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":120,"threads_started":1}
I20260805 23:15:44.486490 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29620
2026-08-05T23:15:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (374 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:44 dist-test-slave-vkjp krb5kdc[29732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:44 dist-test-slave-vkjp krb5kdc[29732](info): set up 2 sockets
Aug 05 23:15:44 dist-test-slave-vkjp krb5kdc[29732](info): commencing operation
krb5kdc: starting...
W20260805 23:15:45.833192 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:45 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:45.969960 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36637
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:36637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:46.087214 29746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:46.087534 29746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:46.087579 29746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:46.091671 29746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:46.091738 29746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:46.091758 29746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:46.091775 29746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:46.091791 29746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:46.097281 29746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:36637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:46.098369 29746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:46.099356 29746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:46.107894 29755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.107904 29753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:46.108075 29746 server_base.cc:1034] running on GCE node
W20260805 23:15:46.107904 29752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:46.108856 29746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:46.110036 29746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:46.111217 29746 hybrid_clock.cc:648] HybridClock initialized: now 1785971746111206 us; error 34 us; skew 500 ppm
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:46.114156 29746 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:46.115285 29746 webserver.cc:460] Webserver started at http://127.23.88.126:44211/ using document root <none> and password file <none>
I20260805 23:15:46.115924 29746 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:46.116031 29746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:46.116295 29746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:46.118597 29746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "699ccf327eed4a7b9e2207685e1cd724"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "bf31c5c2e1adfa60177468d32b7021b2"
server_key_iv: "450bb86cc5147196b942668c0eddd0fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.119364 29746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "699ccf327eed4a7b9e2207685e1cd724"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "bf31c5c2e1adfa60177468d32b7021b2"
server_key_iv: "450bb86cc5147196b942668c0eddd0fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.123764 29746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260805 23:15:46.126562 29762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.127931 29746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:46.128103 29746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "699ccf327eed4a7b9e2207685e1cd724"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "bf31c5c2e1adfa60177468d32b7021b2"
server_key_iv: "450bb86cc5147196b942668c0eddd0fc"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.128229 29746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:46.154328 29746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:46.157850 29746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:46.158087 29746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:46.165680 29746 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:36637
I20260805 23:15:46.165696 29818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:36637 every 8 connection(s)
I20260805 23:15:46.166680 29746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:46.166996 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29746
I20260805 23:15:46.167163 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:46.167521 23905 external_mini_cluster.cc:1428] Setting key 951befe8cb87d04a3d5e42f9015a0b98
I20260805 23:15:46.169718 29819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:46.175462 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724: Bootstrap starting.
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:46.178073 29819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:46.178853 29819 log.cc:824] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:46.180944 29819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724: No bootstrap required, opened a new log
I20260805 23:15:46.181967 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.169022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.23.88.126:36637)
0805 23:15:46.169433 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:46.169459 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:46.169496 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:15:46.170202 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:46.170219 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:46.170546 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:46.170883 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:46.171734 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.172799 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.173642 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.174122 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.174300 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.177045 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:46.177050 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:46.177068 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:46.177100 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:46.179019 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.179514 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.179520 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.179527 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.179649 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.179950 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.179954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.179956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.180363 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:46.180573 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:46.180863 (+ 290us) server_negotiation.cc:299] Negotiation successful
0805 23:15:46.181191 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":137,"threads_started":1}
I20260805 23:15:46.184391 29819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } }
I20260805 23:15:46.184682 29819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:46.184763 29819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 699ccf327eed4a7b9e2207685e1cd724, State: Initialized, Role: FOLLOWER
I20260805 23:15:46.185212 29819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } }
I20260805 23:15:46.185339 29819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:46.185387 29819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:46.185465 29819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:46.186353 29819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } }
I20260805 23:15:46.186650 29819 leader_election.cc:302] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 699ccf327eed4a7b9e2207685e1cd724; no voters:
I20260805 23:15:46.186924 29819 leader_election.cc:288] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:46.187076 29824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:46.187482 29824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [term 1 LEADER]: Becoming Leader. State: Replica: 699ccf327eed4a7b9e2207685e1cd724, State: Running, Role: LEADER
I20260805 23:15:46.187834 29824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } }
I20260805 23:15:46.188561 29819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:46.189594 29826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 699ccf327eed4a7b9e2207685e1cd724. Latest consensus state: current_term: 1 leader_uuid: "699ccf327eed4a7b9e2207685e1cd724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } } }
I20260805 23:15:46.189733 29826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:46.190223 29833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:46.190557 29825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "699ccf327eed4a7b9e2207685e1cd724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "699ccf327eed4a7b9e2207685e1cd724" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36637 } } }
I20260805 23:15:46.191043 29825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:46.193768 29833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:46.200433 29833 catalog_manager.cc:1347] Generated new cluster ID: 09f4d10be221427d96814884a4984292
I20260805 23:15:46.200533 29833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:46.221786 29833 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:46.222767 29833 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:46.235438 29833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 699ccf327eed4a7b9e2207685e1cd724: Generated new TSK 0
I20260805 23:15:46.236436 29833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:46.279281 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:46.394172 29847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:46.394474 29847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:46.394520 29847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:46.398726 29847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:46.398801 29847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:46.398864 29847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:46.404295 29847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:46.405500 29847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:46.406545 29847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:46.418001 29852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.418001 29855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.417999 29853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:46.418511 29847 server_base.cc:1034] running on GCE node
I20260805 23:15:46.418923 29847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:46.419646 29847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:46.420809 29847 hybrid_clock.cc:648] HybridClock initialized: now 1785971746420785 us; error 36 us; skew 500 ppm
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:46.423877 29847 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:46.424742 29847 webserver.cc:460] Webserver started at http://127.23.88.65:37221/ using document root <none> and password file <none>
I20260805 23:15:46.425285 29847 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:46.425346 29847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:46.425608 29847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:46.427871 29847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "ff09fa091b5f4033b7e969cf45f6d9fc"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "20208358a8b9b64bed3487830b61158a"
server_key_iv: "4b96448eb934e24e4278bbb657ac20f5"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.428448 29847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "ff09fa091b5f4033b7e969cf45f6d9fc"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "20208358a8b9b64bed3487830b61158a"
server_key_iv: "4b96448eb934e24e4278bbb657ac20f5"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.432361 29847 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:46.434958 29863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.436311 29847 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:15:46.436445 29847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ff09fa091b5f4033b7e969cf45f6d9fc"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "20208358a8b9b64bed3487830b61158a"
server_key_iv: "4b96448eb934e24e4278bbb657ac20f5"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.436573 29847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:46.459039 29847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:46.462267 29847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:46.462491 29847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:46.463191 29847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:46.464046 29847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:46.464092 29847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.464159 29847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:46.464190 29847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.473310 29847 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:43909
I20260805 23:15:46.473331 29984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:43909 every 8 connection(s)
I20260805 23:15:46.474248 29847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:46.476361 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29847
I20260805 23:15:46.476498 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:46.476795 23905 external_mini_cluster.cc:1428] Setting key 0a0aa97282939c61c71eada9214b3fa0
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:46.489543 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.476137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:57149 (local address 127.23.88.126:36637)
0805 23:15:46.476319 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:46.476324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:46.477339 (+ 1015us) server_negotiation.cc:407] Connection header received
0805 23:15:46.478841 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:46.478844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:46.478900 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:46.478987 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:46.480221 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.480773 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.481980 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.482228 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.482345 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.485231 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:46.485234 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:46.485237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:46.485262 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:46.486839 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.487479 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.487482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.487484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.487562 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.487955 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.487957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.487958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.488143 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:46.488229 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:46.489110 (+ 881us) server_negotiation.cc:299] Negotiation successful
0805 23:15:46.489282 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:15:46.490515 29987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.476606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36637 (local address 127.23.88.65:57149)
0805 23:15:46.477179 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:46.477236 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:46.478618 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:46.479200 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:46.479207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:46.479576 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:46.480064 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.480078 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.480915 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.480919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.481841 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.481850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.482396 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.482401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.482580 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.483304 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:46.483321 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:46.485014 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:46.486972 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.486977 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.486992 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.487384 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.487675 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.487677 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.487679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.487849 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.488322 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:46.488330 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:46.488755 (+ 425us) client_negotiation.cc:769] Sending connection context
0805 23:15:46.488998 (+ 243us) client_negotiation.cc:240] Negotiation successful
0805 23:15:46.489354 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":185,"threads_started":1}
I20260805 23:15:46.492023 29985 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36637
I20260805 23:15:46.492314 29985 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:46.492924 29985 heartbeater.cc:511] Master 127.23.88.126:36637 requested a full tablet report, sending...
I20260805 23:15:46.494602 29781 ts_manager.cc:194] Registered new tserver with Master: ff09fa091b5f4033b7e969cf45f6d9fc (127.23.88.65:43909)
I20260805 23:15:46.495842 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:57149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:46.506893 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:46.620046 29990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:46.620369 29990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:46.620455 29990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:46.624585 29990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:46.624691 29990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:46.624789 29990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:46.630245 29990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:46.631429 29990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:46.632596 29990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:46.642786 29990 server_base.cc:1034] running on GCE node
W20260805 23:15:46.642777 29997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.642777 29999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.643021 29996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:46.643520 29990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:46.644191 29990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:46.645390 29990 hybrid_clock.cc:648] HybridClock initialized: now 1785971746645374 us; error 27 us; skew 500 ppm
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:46.648348 29990 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:46.649410 29990 webserver.cc:460] Webserver started at http://127.23.88.66:45055/ using document root <none> and password file <none>
I20260805 23:15:46.649996 29990 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:46.650089 29990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:46.650327 29990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:46.652484 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "2a72e6a4ff354138a18088623ea22c75"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "a268c323ee1ce6303cf8016cce02b21b"
server_key_iv: "2cfdafb2d828e74cb13a833fafca775f"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.653074 29990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "2a72e6a4ff354138a18088623ea22c75"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "a268c323ee1ce6303cf8016cce02b21b"
server_key_iv: "2cfdafb2d828e74cb13a833fafca775f"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.656908 29990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:46.659439 30006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.660720 29990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:15:46.660861 29990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "2a72e6a4ff354138a18088623ea22c75"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "a268c323ee1ce6303cf8016cce02b21b"
server_key_iv: "2cfdafb2d828e74cb13a833fafca775f"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.660981 29990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:46.692860 29990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:46.696544 29990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:46.696762 29990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:46.697494 29990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:46.698475 29990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:46.698522 29990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.698571 29990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:46.698590 29990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.708278 29990 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:33917
I20260805 23:15:46.708463 30124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:33917 every 8 connection(s)
I20260805 23:15:46.709215 29990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:46.714192 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 29990
I20260805 23:15:46.714298 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:46.714594 23905 external_mini_cluster.cc:1428] Setting key 8842e909c436cc1a16d22b46e4289831
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:46.723289 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.711092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:44087 (local address 127.23.88.126:36637)
0805 23:15:46.711237 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:46.711241 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:46.712060 (+ 819us) server_negotiation.cc:407] Connection header received
0805 23:15:46.713445 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:46.713448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:46.713508 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:46.713591 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:46.714901 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.715527 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.716747 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.716982 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.717089 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.719648 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:46.719652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:46.719657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:46.719686 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:46.721146 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.721686 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.721695 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.721696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.721771 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.722123 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.722127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.722129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.722318 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:46.722425 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:46.722948 (+ 523us) server_negotiation.cc:299] Negotiation successful
0805 23:15:46.723115 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:46.724164 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.711462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36637 (local address 127.23.88.66:44087)
0805 23:15:46.711902 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:46.711944 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:46.713207 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:46.713764 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:46.713770 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:46.714132 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:46.714745 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.714761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.715665 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.715668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.716616 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.716625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.717148 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.717152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.717320 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.717960 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:46.717976 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:46.719471 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:46.721272 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.721278 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.721295 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.721572 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.721864 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.721869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.721871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.722023 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.722427 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:46.722434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:46.722707 (+ 273us) client_negotiation.cc:769] Sending connection context
0805 23:15:46.722904 (+ 197us) client_negotiation.cc:240] Negotiation successful
0805 23:15:46.723271 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":121,"threads_started":1}
I20260805 23:15:46.725454 30125 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36637
I20260805 23:15:46.725728 30125 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:46.726267 30125 heartbeater.cc:511] Master 127.23.88.126:36637 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:15:46.727445 29781 ts_manager.cc:194] Registered new tserver with Master: 2a72e6a4ff354138a18088623ea22c75 (127.23.88.66:33917)
I20260805 23:15:46.728264 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:44087
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:46.746964 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:46.861869 30130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:46.862172 30130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:46.862255 30130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:46.866724 30130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:46.866812 30130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:46.866923 30130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:46.872574 30130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:36637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:46.873840 30130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:46.874913 30130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:46.884887 30136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.884944 30138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:46.884948 30135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:46.885174 30130 server_base.cc:1034] running on GCE node
I20260805 23:15:46.885777 30130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:46.886431 30130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:46.887624 30130 hybrid_clock.cc:648] HybridClock initialized: now 1785971746887607 us; error 26 us; skew 500 ppm
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:46.890700 30130 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:46.891742 30130 webserver.cc:460] Webserver started at http://127.23.88.67:37327/ using document root <none> and password file <none>
I20260805 23:15:46.892321 30130 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:46.892413 30130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:46.892660 30130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:46.894804 30130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "838c2a4b26184b5dac484d7bc96cb5c8"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "7570b4a0699557b1e3d9ec36d68b1d08"
server_key_iv: "ed1bd14ac5c5414999703cc66f7cc15e"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.895493 30130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "838c2a4b26184b5dac484d7bc96cb5c8"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "7570b4a0699557b1e3d9ec36d68b1d08"
server_key_iv: "ed1bd14ac5c5414999703cc66f7cc15e"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.899519 30130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:46.902127 30146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.903541 30130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:46.903681 30130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "838c2a4b26184b5dac484d7bc96cb5c8"
format_stamp: "Formatted at 2026-08-05 23:15:46 on dist-test-slave-vkjp"
server_key: "7570b4a0699557b1e3d9ec36d68b1d08"
server_key_iv: "ed1bd14ac5c5414999703cc66f7cc15e"
server_key_version: "encryptionkey@0"
I20260805 23:15:46.903800 30130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:46.941898 30130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:46.945158 30130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:46.945374 30130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:46.946008 30130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:46.946872 30130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:46.946951 30130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.947028 30130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:46.947080 30130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:46.956091 30130 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35517
I20260805 23:15:46.956108 30264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35517 every 8 connection(s)
I20260805 23:15:46.957059 30130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:46.964238 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30130
I20260805 23:15:46.964483 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:46.964797 23905 external_mini_cluster.cc:1428] Setting key 5f5a9e8a43bf7d9bc9f3c61cfca13722
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:46.971182 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.958931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:52037 (local address 127.23.88.126:36637)
0805 23:15:46.959077 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:46.959120 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:46.959916 (+ 796us) server_negotiation.cc:407] Connection header received
0805 23:15:46.961410 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:46.961414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:46.961491 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:46.961594 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:46.962798 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.963400 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.964607 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.964890 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.964977 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.967934 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:46.967937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:46.967939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:46.967973 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:46.969223 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.969716 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.969719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.969720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.969815 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:46.970136 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:46.970138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:46.970139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:46.970316 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:46.970390 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:46.970877 (+ 487us) server_negotiation.cc:299] Negotiation successful
0805 23:15:46.970993 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:15:46.972092 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:46.959321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:36637 (local address 127.23.88.67:52037)
0805 23:15:46.959763 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:46.959805 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:46.961183 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:46.961779 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:46.961786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:46.962159 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:46.962646 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.962659 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.963534 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.963537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.964406 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:46.964415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:46.965189 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:46.965194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:46.965403 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:46.966126 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:46.966156 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:46.967779 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:46.969330 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.969335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.969348 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.969613 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.969901 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:46.969903 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:46.969912 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:46.970053 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:46.970403 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:46.970408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:46.970663 (+ 255us) client_negotiation.cc:769] Sending connection context
0805 23:15:46.970827 (+ 164us) client_negotiation.cc:240] Negotiation successful
0805 23:15:46.971175 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":119,"threads_started":1}
I20260805 23:15:46.973281 30265 heartbeater.cc:347] Connected to a master server at 127.23.88.126:36637
I20260805 23:15:46.973538 30265 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:46.974123 30265 heartbeater.cc:511] Master 127.23.88.126:36637 requested a full tablet report, sending...
I20260805 23:15:46.975183 29781 ts_manager.cc:194] Registered new tserver with Master: 838c2a4b26184b5dac484d7bc96cb5c8 (127.23.88.67:35517)
I20260805 23:15:46.975965 29781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:52037
I20260805 23:15:46.978909 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/test-admin.keytab.
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:46.996134 23905 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260805 23:15:46.999068 30273 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:46.997645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.23.88.65:43909)
0805 23:15:46.997995 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:46.998001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:46.998026 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:15:46.998081 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:46.998086 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:46.998266 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:46.998389 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:46.998891 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50784: BlockingRecv error: recv got EOF from 127.0.0.1:50784 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
W20260805 23:15:46.999461 23905 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc' of type FILE
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.000519 23905 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:15:46 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.001551 23905 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:47.011391 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:47.002554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.23.88.65:43909)
0805 23:15:47.002694 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:47.002698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:47.002712 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:47.002823 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:47.002825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:47.002868 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:47.002939 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:47.003533 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.004281 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.005087 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.005476 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.005563 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:47.008000 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:47.008011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:47.008024 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:47.008049 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:47.009713 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.010132 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.010137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.010138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.010215 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.010464 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.010467 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.010468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.010725 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:47.010883 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:47.011071 (+ 188us) server_negotiation.cc:299] Negotiation successful
0805 23:15:47.011266 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260805 23:15:47.015784 23905 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc' of type FILE
W20260805 23:15:47.015798 30273 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:47.014509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.23.88.65:43909)
0805 23:15:47.014644 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:47.014648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:47.014693 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:15:47.014802 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:47.014804 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:47.014842 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:47.014905 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:47.015667 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50804: BlockingRecv error: recv got EOF from 127.0.0.1:50804 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.016897 23905 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.017822 23905 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:47.026690 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:47.018750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.23.88.65:43909)
0805 23:15:47.018887 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:47.018891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:47.018985 (+ 94us) server_negotiation.cc:407] Connection header received
0805 23:15:47.019212 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:47.019214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:47.019254 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:47.019340 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:47.019932 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.020454 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.021293 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.021519 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.021630 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:47.023995 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:47.023998 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:47.024001 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:47.024029 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:47.025310 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.025714 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.025717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.025718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.025793 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.026015 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.026016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.026018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.026176 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:47.026286 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:47.026447 (+ 161us) server_negotiation.cc:299] Negotiation successful
0805 23:15:47.026562 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260805 23:15:47.029909 30273 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:47.028749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.23.88.65:43909)
0805 23:15:47.028915 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:47.028920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:47.029045 (+ 125us) server_negotiation.cc:407] Connection header received
0805 23:15:47.029204 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:47.029207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:47.029257 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:47.029338 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:47.029767 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50818: BlockingRecv error: recv got EOF from 127.0.0.1:50818 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260805 23:15:47.030052 23905 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785971715702001-23905-0/krb5kdc/krb5cc' of type FILE
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.031181 23905 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:47.032117 23905 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[29732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:15:47.040879 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:47.032995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.23.88.65:43909)
0805 23:15:47.033151 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:47.033156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:47.033209 (+ 53us) server_negotiation.cc:407] Connection header received
0805 23:15:47.033341 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:47.033344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:47.033383 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:47.033452 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:47.033965 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.034466 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.035282 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:47.035500 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:47.035579 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:47.037780 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:47.037784 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:47.037786 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:47.037812 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:47.039068 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.039600 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.039603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.039605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.039693 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:47.039980 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:47.039984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:47.039986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:47.040193 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:47.040360 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:47.040601 (+ 241us) server_negotiation.cc:299] Negotiation successful
0805 23:15:47.040715 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:47.042268 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29847
I20260805 23:15:47.049328 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29990
I20260805 23:15:47.055763 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30130
I20260805 23:15:47.064057 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 29746
2026-08-05T23:15:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2322 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[30286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[30286](info): set up 2 sockets
Aug 05 23:15:47 dist-test-slave-vkjp krb5kdc[30286](info): commencing operation
krb5kdc: starting...
W20260805 23:15:48.142498 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab.
Entry for principal oryx/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab.
I20260805 23:15:48.274894 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:46333
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab
--principal=oryx/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:46333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:48.391577 30300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:48.391877 30300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:48.391921 30300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:48.396052 30300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:48.396126 30300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:48.396147 30300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:48.396163 30300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:48.396179 30300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:48.401948 30300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:46333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab
--principal=oryx/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:48.403132 30300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:48.404145 30300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:48.413188 30309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.413347 30300 server_base.cc:1034] running on GCE node
W20260805 23:15:48.413170 30306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:48.413170 30307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.414294 30300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:48.415578 30300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:48.416756 30300 hybrid_clock.cc:648] HybridClock initialized: now 1785971748416730 us; error 50 us; skew 500 ppm
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:48.424652 30300 init.cc:377] Logged in from keytab as oryx/127.23.88.126@KRBTEST.COM (short username oryx)
I20260805 23:15:48.425825 30300 webserver.cc:460] Webserver started at http://127.23.88.126:43683/ using document root <none> and password file <none>
I20260805 23:15:48.426416 30300 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:48.426481 30300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:48.426661 30300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:48.428877 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "cda1e5ea711f4b7a879662344e5df31d"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "5d081d33b4d1eb200b2732f52f6d4a27"
server_key_iv: "b9da9afd9a2113603265c80b5963309b"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.429443 30300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "cda1e5ea711f4b7a879662344e5df31d"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "5d081d33b4d1eb200b2732f52f6d4a27"
server_key_iv: "b9da9afd9a2113603265c80b5963309b"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.433629 30300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:15:48.436302 30316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:48.437806 30300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:15:48.437995 30300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "cda1e5ea711f4b7a879662344e5df31d"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "5d081d33b4d1eb200b2732f52f6d4a27"
server_key_iv: "b9da9afd9a2113603265c80b5963309b"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.438136 30300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:48.457588 30300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:48.461383 30300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:48.461645 30300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:48.470319 30300 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:46333
I20260805 23:15:48.470429 30368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:46333 every 8 connection(s)
I20260805 23:15:48.471487 30300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:48.472529 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30300
I20260805 23:15:48.472680 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:48.472985 23905 external_mini_cluster.cc:1428] Setting key 772237199efbc10a210d18df0547600d
I20260805 23:15:48.475157 30369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:48.482805 30369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Bootstrap starting.
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:48.486080 30369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:48.487075 30369 log.cc:824] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:48.489399 30369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: No bootstrap required, opened a new log
I20260805 23:15:48.489480 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:48.474524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.23.88.126:46333)
0805 23:15:48.474983 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:48.475004 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:48.475051 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:15:48.476012 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:48.476037 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:48.476479 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:48.476887 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:48.477612 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.479376 (+ 1764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.480270 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.481078 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.481278 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:48.483999 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:48.484006 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:48.484024 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:48.484067 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:48.486231 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:48.486680 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:48.486686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:48.486692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:48.486866 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:48.487170 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:48.487174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:48.487176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:48.487763 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:48.488006 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:48.488343 (+ 337us) server_negotiation.cc:299] Negotiation successful
0805 23:15:48.488677 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":109,"threads_started":1}
I20260805 23:15:48.493260 30369 raft_consensus.cc:348] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } }
I20260805 23:15:48.493541 30369 raft_consensus.cc:374] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:48.493636 30369 raft_consensus.cc:729] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cda1e5ea711f4b7a879662344e5df31d, State: Initialized, Role: FOLLOWER
I20260805 23:15:48.494081 30369 consensus_queue.cc:260] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } }
I20260805 23:15:48.494243 30369 raft_consensus.cc:388] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:48.494333 30369 raft_consensus.cc:482] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:48.494462 30369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:48.495805 30369 raft_consensus.cc:504] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } }
I20260805 23:15:48.496163 30369 leader_election.cc:302] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cda1e5ea711f4b7a879662344e5df31d; no voters:
I20260805 23:15:48.496502 30369 leader_election.cc:288] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:48.496650 30376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:48.496909 30376 raft_consensus.cc:686] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 LEADER]: Becoming Leader. State: Replica: cda1e5ea711f4b7a879662344e5df31d, State: Running, Role: LEADER
I20260805 23:15:48.497259 30376 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } }
I20260805 23:15:48.497829 30369 sys_catalog.cc:564] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:48.499307 30377 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } }
I20260805 23:15:48.499431 30377 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:48.499711 30378 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cda1e5ea711f4b7a879662344e5df31d. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } }
I20260805 23:15:48.499805 30378 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
W20260805 23:15:48.502897 30393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:48.503007 30393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:48.503075 30387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:48.503979 30387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:48.509601 30387 catalog_manager.cc:1347] Generated new cluster ID: 84c7ee6b29114756b0299c5d6d2c4439
I20260805 23:15:48.509696 30387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:48.523140 30387 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:48.524536 30387 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:48.535925 30387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Generated new TSK 0
I20260805 23:15:48.536878 30387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
I20260805 23:15:48.582599 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:48.701277 30399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:48.701589 30399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:48.701637 30399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:48.705801 30399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:48.705890 30399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:48.705955 30399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:48.711669 30399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:48.712796 30399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:48.713807 30399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:48.724436 30405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:48.724438 30404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.724485 30399 server_base.cc:1034] running on GCE node
W20260805 23:15:48.724436 30407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.725261 30399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:48.726022 30399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:48.727212 30399 hybrid_clock.cc:648] HybridClock initialized: now 1785971748727180 us; error 54 us; skew 500 ppm
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:48.730247 30399 init.cc:377] Logged in from keytab as oryx/127.23.88.65@KRBTEST.COM (short username oryx)
I20260805 23:15:48.731302 30399 webserver.cc:460] Webserver started at http://127.23.88.65:35267/ using document root <none> and password file <none>
I20260805 23:15:48.731895 30399 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:48.731963 30399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:48.732141 30399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:48.734292 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "ccbd5600c41f4c7296ad486a288199c4"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "fb1bc1c2ed905d34458baa868005e674"
server_key_iv: "83013a69ddf2edccd571f11a85f9b59a"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.734866 30399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "ccbd5600c41f4c7296ad486a288199c4"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "fb1bc1c2ed905d34458baa868005e674"
server_key_iv: "83013a69ddf2edccd571f11a85f9b59a"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.739501 30399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:48.742270 30414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:48.743773 30399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:48.743887 30399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ccbd5600c41f4c7296ad486a288199c4"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "fb1bc1c2ed905d34458baa868005e674"
server_key_iv: "83013a69ddf2edccd571f11a85f9b59a"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.743995 30399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:48.776875 30399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:48.780668 30399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:48.780846 30399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:48.781455 30399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:48.782541 30399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:48.782612 30399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:48.782680 30399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:48.782733 30399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:48.793089 30399 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:45877
I20260805 23:15:48.793115 30534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:45877 every 8 connection(s)
I20260805 23:15:48.794076 30399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:48.800473 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30399
I20260805 23:15:48.800621 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:48.800943 23905 external_mini_cluster.cc:1428] Setting key d131ebe8c7ba771e6fa180acaa2fcc5e
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:48.810328 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:48.796023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:32897 (local address 127.23.88.126:46333)
0805 23:15:48.796175 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:48.796178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:48.797063 (+ 885us) server_negotiation.cc:407] Connection header received
0805 23:15:48.798456 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:48.798460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:48.798550 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:48.798656 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:48.800207 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.801077 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.802487 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.802722 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.802844 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:48.806021 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:48.806024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:48.806026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:48.806053 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:48.807676 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:48.808434 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:48.808437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:48.808439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:48.808521 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:48.808998 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:48.809001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:48.809003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:48.809227 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:48.809318 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:48.810072 (+ 754us) server_negotiation.cc:299] Negotiation successful
0805 23:15:48.810200 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for oryx/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:48.811705 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:48.796419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:32897)
0805 23:15:48.796879 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:48.796928 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:48.798239 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:48.798905 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:48.798912 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:48.799395 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:48.800008 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:48.800024 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.801216 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.801219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:48.802355 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:48.802364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:48.802892 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:48.802896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:48.803144 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:48.803802 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:48.803819 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:48.805813 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:48.807825 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:48.807835 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:48.807858 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:48.808313 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:48.808625 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:48.808631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:48.808634 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:48.808864 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:48.809353 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:48.809363 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:48.809775 (+ 412us) client_negotiation.cc:769] Sending connection context
0805 23:15:48.810026 (+ 251us) client_negotiation.cc:240] Negotiation successful
0805 23:15:48.810436 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":126,"threads_started":1}
I20260805 23:15:48.813241 30535 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:48.813493 30535 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:48.814103 30535 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:48.816109 30333 ts_manager.cc:194] Registered new tserver with Master: ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877)
I20260805 23:15:48.817379 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:32897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
I20260805 23:15:48.834085 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:48.950636 30540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:48.950947 30540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:48.951020 30540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:48.955446 30540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:48.955526 30540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:48.955626 30540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:48.961506 30540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:48.962746 30540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:48.963907 30540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:48.974136 30546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:48.974166 30545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.974257 30540 server_base.cc:1034] running on GCE node
W20260805 23:15:48.974159 30548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:48.974890 30540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:48.975687 30540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:48.976891 30540 hybrid_clock.cc:648] HybridClock initialized: now 1785971748976876 us; error 25 us; skew 500 ppm
Aug 05 23:15:48 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:48.979960 30540 init.cc:377] Logged in from keytab as oryx/127.23.88.66@KRBTEST.COM (short username oryx)
I20260805 23:15:48.980947 30540 webserver.cc:460] Webserver started at http://127.23.88.66:35169/ using document root <none> and password file <none>
I20260805 23:15:48.981544 30540 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:48.981638 30540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:48.981885 30540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:48.984184 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "0454c331efec2f63f0ca8cad84f61c1f"
server_key_iv: "a2914770d03d5bdbaf6db0bcc1f79341"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.984794 30540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "0454c331efec2f63f0ca8cad84f61c1f"
server_key_iv: "a2914770d03d5bdbaf6db0bcc1f79341"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.989271 30540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:15:48.992009 30555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:48.993373 30540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:48.993539 30540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "0454c331efec2f63f0ca8cad84f61c1f"
server_key_iv: "a2914770d03d5bdbaf6db0bcc1f79341"
server_key_version: "encryptionkey@0"
I20260805 23:15:48.993672 30540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.021087 30540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:49.025106 30540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:49.025338 30540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:49.025991 30540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:49.026875 30540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:49.026957 30540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.027027 30540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:49.027076 30540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.036278 30540 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:46667
I20260805 23:15:49.036334 30673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:46667 every 8 connection(s)
I20260805 23:15:49.037237 30540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:49.041569 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30540
I20260805 23:15:49.041734 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:49.042111 23905 external_mini_cluster.cc:1428] Setting key 2e7ee91bc5c60549dae0a687aedc3635
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:49.053663 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.039161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46065 (local address 127.23.88.126:46333)
0805 23:15:49.039323 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.039327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.040218 (+ 891us) server_negotiation.cc:407] Connection header received
0805 23:15:49.041924 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.041928 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.041989 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.042068 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.043608 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.044172 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.046050 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.046453 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.046581 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.049714 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.049718 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.049720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.049748 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.051391 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.051980 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.051983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.051984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.052065 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.052434 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.052437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.052439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.052685 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.052800 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.053349 (+ 549us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.053487 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:15:49.054536 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.039574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:46065)
0805 23:15:49.040039 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.040095 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.041631 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.042291 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.042300 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.042749 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.043435 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.043457 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.044313 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.044317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.045909 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.045921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.046625 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.046628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.046793 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.047610 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.047637 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.049502 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.051526 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.051533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.051552 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.051847 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.052161 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.052165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.052167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.052327 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.052799 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.052806 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.053090 (+ 284us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.053289 (+ 199us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.053661 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.67@KRBTEST.COM" created.
I20260805 23:15:49.055858 30674 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:49.056133 30674 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:49.056663 30674 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
I20260805 23:15:49.057787 30333 ts_manager.cc:194] Registered new tserver with Master: e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:49.058655 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:46065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
I20260805 23:15:49.075237 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:49.192184 30679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:49.192482 30679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:49.192529 30679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:49.196748 30679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:49.196818 30679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:49.196902 30679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:49.202847 30679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:46333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:49.203991 30679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:49.204983 30679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:49.215165 30685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.215165 30684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:49.215246 30679 server_base.cc:1034] running on GCE node
W20260805 23:15:49.215190 30687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:49.216029 30679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:49.216657 30679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:49.217845 30679 hybrid_clock.cc:648] HybridClock initialized: now 1785971749217820 us; error 36 us; skew 500 ppm
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:49.220857 30679 init.cc:377] Logged in from keytab as oryx/127.23.88.67@KRBTEST.COM (short username oryx)
I20260805 23:15:49.221836 30679 webserver.cc:460] Webserver started at http://127.23.88.67:44503/ using document root <none> and password file <none>
I20260805 23:15:49.222429 30679 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:49.222525 30679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:49.222779 30679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:49.224996 30679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "140d6d27f0214b86a6211f44f74aa97e"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "4566ca225e0105376a2dc6e1a42a5a54"
server_key_iv: "6f2f9363fb505e1d4ab61e2dcf4856fe"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.225595 30679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "140d6d27f0214b86a6211f44f74aa97e"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "4566ca225e0105376a2dc6e1a42a5a54"
server_key_iv: "6f2f9363fb505e1d4ab61e2dcf4856fe"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.229702 30679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:49.232524 30694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.233913 30679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:15:49.234081 30679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "140d6d27f0214b86a6211f44f74aa97e"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "4566ca225e0105376a2dc6e1a42a5a54"
server_key_iv: "6f2f9363fb505e1d4ab61e2dcf4856fe"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.234215 30679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.257934 30679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:49.261622 30679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:49.261863 30679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:49.262574 30679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:49.263520 30679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:49.263592 30679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.263664 30679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:49.263715 30679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.273056 30679 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35477
I20260805 23:15:49.273077 30814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35477 every 8 connection(s)
I20260805 23:15:49.274039 30679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:49.282742 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30679
I20260805 23:15:49.282896 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:49.283332 23905 external_mini_cluster.cc:1428] Setting key 6f4ce008742b2f1d4007eccb8e00707e
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:49.288841 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.276298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:51671 (local address 127.23.88.126:46333)
0805 23:15:49.276497 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.276503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.277477 (+ 974us) server_negotiation.cc:407] Connection header received
0805 23:15:49.278815 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.278818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.278871 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.278954 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.280281 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.280873 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.282140 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.282402 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.282525 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.285553 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.285556 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.285559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.285593 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.286844 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.287379 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.287382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.287383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.287463 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.287783 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.287785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.287786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.287983 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.288091 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.288602 (+ 511us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.288715 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:15:49.289817 30818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.276753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:51671)
0805 23:15:49.277269 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.277317 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.278601 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.279203 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.279210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.279589 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.280101 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.280114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.281080 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.281083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.282016 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.282026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.282582 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.282585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.282730 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.283499 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.283523 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.285311 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.286955 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.286961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.286978 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.287281 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.287550 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.287552 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.287553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.287694 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.288086 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.288092 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.288347 (+ 255us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.288532 (+ 185us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.288812 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":148,"threads_started":1}
I20260805 23:15:49.290979 30816 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:49.291271 30816 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:49.291821 30816 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
I20260805 23:15:49.292933 30333 ts_manager.cc:194] Registered new tserver with Master: 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:49.293493 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:51671
I20260805 23:15:49.297423 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.88.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.125.keytab.
Entry for principal oryx/127.23.88.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.125.keytab.
I20260805 23:15:49.326442 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:40405
--webserver_interface=127.23.88.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.125.keytab
--principal=oryx/127.23.88.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:46333,127.23.88.125:40405
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:49.443338 30822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:49.443688 30822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:49.443781 30822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:49.448153 30822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:49.448246 30822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:49.448282 30822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:49.448348 30822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:49.448386 30822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:49.454067 30822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:46333,127.23.88.125:40405
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.125.keytab
--principal=oryx/127.23.88.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:40405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.88.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
W20260805 23:15:49.454380 30822 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.88.126:46333,127.23.88.125:40405'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260805 23:15:49.455389 30822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:49.456489 30822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:49.465402 30822 server_base.cc:1034] running on GCE node
W20260805 23:15:49.465411 30830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.465341 30827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.465337 30828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:49.466212 30822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:49.467479 30822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:49.468695 30822 hybrid_clock.cc:648] HybridClock initialized: now 1785971749468667 us; error 45 us; skew 500 ppm
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:49.471827 30822 init.cc:377] Logged in from keytab as oryx/127.23.88.125@KRBTEST.COM (short username oryx)
I20260805 23:15:49.472971 30822 webserver.cc:460] Webserver started at http://127.23.88.125:43127/ using document root <none> and password file <none>
I20260805 23:15:49.473640 30822 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:49.473721 30822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:49.473954 30822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:49.476655 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/instance:
uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "cdd58ab37bdc02253fda8c51a91f8c79"
server_key_iv: "75980131bfeaa0c0e42f72bc8495ebf1"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.477331 30822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal/instance:
uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "cdd58ab37bdc02253fda8c51a91f8c79"
server_key_iv: "75980131bfeaa0c0e42f72bc8495ebf1"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.482292 30822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260805 23:15:49.485491 30837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.486855 30822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:15:49.487021 30822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "cdd58ab37bdc02253fda8c51a91f8c79"
server_key_iv: "75980131bfeaa0c0e42f72bc8495ebf1"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.487216 30822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.507457 30822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:49.510378 30822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:49.510608 30822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:49.518110 30822 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.125:40405
I20260805 23:15:49.518435 30892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.125:40405 every 8 connection(s)
I20260805 23:15:49.519228 30822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
I20260805 23:15:49.522295 30893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:49.523772 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30822
I20260805 23:15:49.527356 30893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.125@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:49.538995 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.529146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.23.88.126:46333)
0805 23:15:49.529299 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.529303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.529617 (+ 314us) server_negotiation.cc:407] Connection header received
0805 23:15:49.529784 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.529795 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.529841 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.529935 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.530588 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.531181 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.532081 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.532361 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.532474 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.535339 (+ 2865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.535343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.535346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.535378 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.536969 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537433 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.537437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.537438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.537528 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537866 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.537870 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.537872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.538099 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.538211 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.538614 (+ 403us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.538787 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:15:49.539835 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.529084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34616)
0805 23:15:49.529494 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.529532 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.529674 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.529981 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.529986 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.530331 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.530462 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.530467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.531313 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.531316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.531953 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.531961 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.532515 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.532519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.532627 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.533158 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.533173 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.535190 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.537080 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537089 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.537099 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.537318 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.537680 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.537684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.537769 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.538211 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.538220 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.538371 (+ 151us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.538553 (+ 182us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.539039 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":116,"thread_start_us":169,"threads_started":1}
I20260805 23:15:49.539839 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.525370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58860 (local address 127.23.88.125:40405)
0805 23:15:49.525827 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.525844 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.525884 (+ 40us) server_negotiation.cc:407] Connection header received
0805 23:15:49.526566 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.526584 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.526934 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.527383 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.528256 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.529825 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.530756 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.531376 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.531553 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.534554 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.534560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.534570 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.534609 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.536668 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537128 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.537139 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.537140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.537250 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.537544 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.537548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.537549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.538045 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.538265 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.538541 (+ 276us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.538945 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":295,"thread_start_us":153,"threads_started":1}
I20260805 23:15:49.541181 30893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.125@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
I20260805 23:15:49.549681 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.541951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58866 (local address 127.23.88.125:40405)
0805 23:15:49.542078 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.542081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.542092 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:15:49.542125 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.542126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.542159 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.542244 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.542981 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.543558 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.544286 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.544531 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.544638 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.546544 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.546548 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.546550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.546572 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.547979 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.548482 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.548487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.548489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.548568 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.548837 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.548842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.548843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.549027 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.549133 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.549390 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.549536 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":59}
I20260805 23:15:49.549681 30897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.541730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58866)
0805 23:15:49.541847 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.541881 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.541968 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.542260 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.542262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.542430 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.542836 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.542842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.543664 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.543669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.544173 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.544180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.544637 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.544640 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.544726 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.545159 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.545170 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.546417 (+ 1247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.548093 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.548095 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.548097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.548387 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.548664 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.548667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.548669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.548751 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.549120 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.549123 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.549204 (+ 81us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.549317 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.549476 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260805 23:15:49.552838 30893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Bootstrap starting.
I20260805 23:15:49.555275 30893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:49.556073 30893 log.cc:824] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:49.558053 30893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: No bootstrap required, opened a new log
I20260805 23:15:49.561085 30893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } }
I20260805 23:15:49.561303 30893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:49.561338 30893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475ff05cb07a4d92a7b4c17b4dc0eedb, State: Initialized, Role: FOLLOWER
I20260805 23:15:49.561715 30893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } }
I20260805 23:15:49.562211 30899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } } }
I20260805 23:15:49.562408 30899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:49.562930 30893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:15:49.568081 30912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:49.568215 30912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:49.570668 30822 master_runner.cc:428] Detected that this master 475ff05cb07a4d92a7b4c17b4dc0eedb is joining an existing cluster
I20260805 23:15:49.570747 30822 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.88.125:40405
I20260805 23:15:49.583235 30919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.573344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58880)
0805 23:15:49.573839 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.573851 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.573999 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.574290 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.574294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.574559 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.574713 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.574722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.575605 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.575610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.576253 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.576261 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.576925 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.576932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.577031 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.578140 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.578160 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.578856 (+ 696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.581363 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.581368 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.581371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.581672 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.582031 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.582037 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.582040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.582127 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.582572 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.582575 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.582678 (+ 103us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.582788 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.582942 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"spinlock_wait_cycles":34304,"thread_start_us":148,"threads_started":1}
I20260805 23:15:49.583240 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.573471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58880 (local address 127.23.88.125:40405)
0805 23:15:49.573773 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.573778 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.573931 (+ 153us) server_negotiation.cc:407] Connection header received
0805 23:15:49.574103 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.574106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.574152 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.574229 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.574827 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.575492 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.576371 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.576782 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.576887 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.579188 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.579192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.579194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.579223 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.581228 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.581790 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.581794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.581796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.581901 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.582205 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.582209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.582211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.582464 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.582566 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.582855 (+ 289us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.583012 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":225}
I20260805 23:15:49.583591 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.573589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.23.88.126:46333)
0805 23:15:49.573722 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.573727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.574321 (+ 594us) server_negotiation.cc:407] Connection header received
0805 23:15:49.574498 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.574506 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.574551 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.574637 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.575182 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.575996 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.576966 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.577312 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.577374 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.579019 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.579023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.579026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.579056 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.581226 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.582181 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.582185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.582187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.582291 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.582627 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.582630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.582632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.582858 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.582954 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.583316 (+ 362us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.583451 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:15:49.584192 30918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.573345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34626)
0805 23:15:49.574240 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.574253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.574379 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.574652 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.574655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.574905 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.575025 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.575031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.576124 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.576128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.576798 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.576806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.577557 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.577560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.577678 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.578140 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.578159 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.578856 (+ 697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.581798 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.581802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.581804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.582074 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.582413 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.582416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.582418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.582500 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.583001 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.583004 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.583119 (+ 115us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.583814 (+ 695us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.583988 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":47,"spinlock_wait_cycles":12288,"thread_start_us":125,"threads_started":1}
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.126@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
I20260805 23:15:49.605072 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.591842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58894)
0805 23:15:49.593665 (+ 1823us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.593741 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.593912 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.594590 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.594599 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.595185 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.595366 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.595375 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.596453 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.596458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.597485 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.597497 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.597999 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.598003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.598137 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.598770 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.598786 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.600705 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.602937 (+ 2232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.602949 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.602957 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.603336 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.603677 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.603680 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.603682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.603784 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.604304 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.604312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.604475 (+ 163us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.604634 (+ 159us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.604872 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":113,"threads_started":1}
I20260805 23:15:49.605456 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.592536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.23.88.125:40405)
0805 23:15:49.592659 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.592664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.594194 (+ 1530us) server_negotiation.cc:407] Connection header received
0805 23:15:49.594272 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.594275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.594336 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.594418 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.595515 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.596295 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.597619 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.597857 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.597982 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.600876 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.600879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.600881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.600907 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.602792 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.603464 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.603468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.603470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.603565 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.603901 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.603904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.603907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.604175 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.604290 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.604753 (+ 463us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.604910 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for oryx/127.23.88.124@KRBTEST.COM; defaulting to no policy
I20260805 23:15:49.607245 30333 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.88.125:40405: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
member_type: NON_VOTER
last_known_addr {
host: "127.23.88.125"
port: 40405
}
attrs {
promote: true
}
}
dest_uuid: "cda1e5ea711f4b7a879662344e5df31d"
cas_config_opid_index: -1
I20260805 23:15:49.607983 30333 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.124@KRBTEST.COM" created.
I20260805 23:15:49.610142 30923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } }
I20260805 23:15:49.610841 30865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:49.612013 30923 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.88.125:40405
I20260805 23:15:49.612332 30865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260805 23:15:49.612381 30923 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } }
I20260805 23:15:49.612476 30923 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:49.612571 30822 master.cc:537] Master@127.23.88.125:40405 shutting down...
I20260805 23:15:49.614101 30924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260805 23:15:49.616438 30319 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260805 23:15:49.617486 30822 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Raft consensus shutting down.
W20260805 23:15:49.617517 30319 consensus_peers.cc:487] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d -> Peer 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125:40405): Couldn't send request to peer 475ff05cb07a4d92a7b4c17b4dc0eedb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:15:49.617796 30822 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:49.617933 30822 tablet_replica.cc:330] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.124.keytab.
Entry for principal oryx/127.23.88.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.124.keytab.
I20260805 23:15:49.630172 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:42685
--webserver_interface=127.23.88.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.124.keytab
--principal=oryx/127.23.88.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:49.634178 30822 master.cc:552] Master@127.23.88.125:40405 shutdown complete.
I20260805 23:15:49.634261 30822 master_runner.cc:305] Clearing existing system tablet
I20260805 23:15:49.634545 30822 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:49.634613 30822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:49.636307 30822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:49.637427 30930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.637740 30822 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.637989 30822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:49.638047 30822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "cdd58ab37bdc02253fda8c51a91f8c79"
server_key_iv: "75980131bfeaa0c0e42f72bc8495ebf1"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.640951 30822 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:49.644223 30822 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 23:15:49.644340 30822 log.cc:1196] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260805 23:15:49.644639 30822 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Deleting consensus metadata
I20260805 23:15:49.644835 30822 master_runner.cc:315] Copying system tablet from 127.23.88.126:46333
I20260805 23:15:49.646273 30822 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: tablet copy: Beginning tablet copy session from remote peer at address 127.23.88.126:46333
I20260805 23:15:49.656055 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.647066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.23.88.126:46333)
0805 23:15:49.647263 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.647268 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.647448 (+ 180us) server_negotiation.cc:407] Connection header received
0805 23:15:49.647603 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.647606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.647662 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.647763 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.648315 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.649179 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.650370 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.650719 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.650815 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.652398 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.652402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.652405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.652435 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.654159 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.654653 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.654656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.654658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.654751 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.655053 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.655057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.655059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.655326 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.655437 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.655755 (+ 318us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.655885 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260805 23:15:49.656085 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.646939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34638)
0805 23:15:49.647361 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.647377 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.647500 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.647802 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.647805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.648032 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.648171 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.648179 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.649335 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.649338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.650244 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.650254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.650965 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.650968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.651071 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.651712 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.651734 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.652226 (+ 492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.654282 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.654286 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.654288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.654547 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.654852 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.654855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.654857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.654955 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.655438 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.655441 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.655551 (+ 110us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.655675 (+ 124us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.655839 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":108,"threads_started":1}
I20260805 23:15:49.657218 30353 tablet_copy_service.cc:140] P cda1e5ea711f4b7a879662344e5df31d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 475ff05cb07a4d92a7b4c17b4dc0eedb ({username='oryx', principal='oryx/127.23.88.125@KRBTEST.COM'} at 127.0.0.1:34638)
I20260805 23:15:49.657437 30353 tablet_copy_service.cc:161] P cda1e5ea711f4b7a879662344e5df31d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 475ff05cb07a4d92a7b4c17b4dc0eedb at {username='oryx', principal='oryx/127.23.88.125@KRBTEST.COM'} at 127.0.0.1:34638: session id = 475ff05cb07a4d92a7b4c17b4dc0eedb-00000000000000000000000000000000
I20260805 23:15:49.659451 30353 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:15:49.661444 30822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 23:15:49.664143 30822 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: tablet copy: Starting download of 0 data blocks...
I20260805 23:15:49.664350 30822 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: tablet copy: Starting download of 1 WAL segments...
I20260805 23:15:49.666419 30822 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:15:49.668552 30353 tablet_copy_service.cc:342] P cda1e5ea711f4b7a879662344e5df31d: Request end of tablet copy session 475ff05cb07a4d92a7b4c17b4dc0eedb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.88.125@KRBTEST.COM'} at 127.0.0.1:34638
I20260805 23:15:49.668701 30353 tablet_copy_service.cc:433] P cda1e5ea711f4b7a879662344e5df31d: ending tablet copy session 475ff05cb07a4d92a7b4c17b4dc0eedb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 475ff05cb07a4d92a7b4c17b4dc0eedb
I20260805 23:15:49.676963 30822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:49.678795 30937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.678814 30940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.678977 30938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:49.679750 30822 server_base.cc:1034] running on GCE node
I20260805 23:15:49.679921 30822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:49.680233 30822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:49.681391 30822 hybrid_clock.cc:648] HybridClock initialized: now 1785971749681378 us; error 32 us; skew 500 ppm
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:49.683157 30822 init.cc:377] Logged in from keytab as oryx/127.23.88.125@KRBTEST.COM (short username oryx)
I20260805 23:15:49.683694 30822 webserver.cc:460] Webserver started at http://127.23.88.125:34885/ using document root <none> and password file <none>
I20260805 23:15:49.683943 30822 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:49.683990 30822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:49.685384 30822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:49.686561 30948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.686851 30822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:49.687063 30822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "cdd58ab37bdc02253fda8c51a91f8c79"
server_key_iv: "75980131bfeaa0c0e42f72bc8495ebf1"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.687271 30822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.713218 30822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:49.715731 30822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:49.720917 30822 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.125:40405
I20260805 23:15:49.720947 31000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.125:40405 every 8 connection(s)
I20260805 23:15:49.721990 30822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
I20260805 23:15:49.724182 31001 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:15:49.725294 31001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Bootstrap starting.
I20260805 23:15:49.736240 31001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:15:49.736599 31001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: Bootstrap complete.
I20260805 23:15:49.737210 31001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } }
I20260805 23:15:49.737368 31001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 475ff05cb07a4d92a7b4c17b4dc0eedb, State: Initialized, Role: LEARNER
I20260805 23:15:49.737480 31001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } }
I20260805 23:15:49.737960 31001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:49.737982 31004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } }
I20260805 23:15:49.738323 31004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: LEARNER
I20260805 23:15:49.741545 31015 catalog_manager.cc:1259] Loaded cluster ID: 84c7ee6b29114756b0299c5d6d2c4439
I20260805 23:15:49.741612 31015 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: loading cluster ID for follower catalog manager: success
I20260805 23:15:49.743956 31015 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: acquiring CA information for follower catalog manager: success
I20260805 23:15:49.745607 31015 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260805 23:15:49.774782 30928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:49.775156 30928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:49.775219 30928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:49.779516 30928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:49.779587 30928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:49.779608 30928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:49.779624 30928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:49.779640 30928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:49.785526 30928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.124.keytab
--principal=oryx/127.23.88.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:42685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.88.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:49.786679 30928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:49.787870 30928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:49.796638 31022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.796645 31024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:49.796657 31021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:49.796864 30928 server_base.cc:1034] running on GCE node
I20260805 23:15:49.797602 30928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:49.798959 30928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:49.800134 30928 hybrid_clock.cc:648] HybridClock initialized: now 1785971749800119 us; error 42 us; skew 500 ppm
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:49.803023 30928 init.cc:377] Logged in from keytab as oryx/127.23.88.124@KRBTEST.COM (short username oryx)
I20260805 23:15:49.804023 30928 webserver.cc:460] Webserver started at http://127.23.88.124:38643/ using document root <none> and password file <none>
I20260805 23:15:49.804574 30928 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:49.804634 30928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:49.804805 30928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:49.806890 30928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/instance:
uuid: "997df6ed0f6d456692e7c05692f5e658"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "aee1e30a2ca3b6eb09a0a3efe9ad3f8d"
server_key_iv: "aa919cbc4f072b991bbbfcadc0be4fe6"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.807520 30928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal/instance:
uuid: "997df6ed0f6d456692e7c05692f5e658"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "aee1e30a2ca3b6eb09a0a3efe9ad3f8d"
server_key_iv: "aa919cbc4f072b991bbbfcadc0be4fe6"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.811551 30928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:15:49.814154 31032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:49.815534 30928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:49.815685 30928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "997df6ed0f6d456692e7c05692f5e658"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "aee1e30a2ca3b6eb09a0a3efe9ad3f8d"
server_key_iv: "aa919cbc4f072b991bbbfcadc0be4fe6"
server_key_version: "encryptionkey@0"
I20260805 23:15:49.815814 30928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:49.820649 30535 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
I20260805 23:15:49.831297 30928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:49.834800 30928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:49.835029 30928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:49.845458 30928 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.124:42685
I20260805 23:15:49.845774 31087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.124:42685 every 8 connection(s)
I20260805 23:15:49.846549 30928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
I20260805 23:15:49.849795 31088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:49.849891 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 30928
I20260805 23:15:49.855145 31088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.124@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:49.866961 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.851724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.23.88.124:42685)
0805 23:15:49.852176 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.852195 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.852239 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:15:49.853361 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.853389 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.853850 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.854241 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.854954 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.856430 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.857526 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.858195 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.858374 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.861218 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.861223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.861233 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.861288 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.863395 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.863938 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.863943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.863948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.864108 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.864391 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.864395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.864397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.865027 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.865248 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.865500 (+ 252us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.865826 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":126,"threads_started":1}
I20260805 23:15:49.867527 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.857516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34652 (local address 127.23.88.126:46333)
0805 23:15:49.857646 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.857651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.858051 (+ 400us) server_negotiation.cc:407] Connection header received
0805 23:15:49.858180 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.858184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.858247 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.858335 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.859233 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.859735 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.860813 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.861208 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.861291 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.864102 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.864106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.864108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.864139 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.865696 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.866151 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.866154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.866156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.866246 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.866526 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.866529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.866531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.866751 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.866851 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.867224 (+ 373us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.867356 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:49.868145 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.857208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34652)
0805 23:15:49.857886 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.857938 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.858081 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.858401 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.858407 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.858885 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.859071 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.859083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.859866 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.859873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.860668 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.860679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.861313 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.861316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.861456 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.862116 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.862136 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.863952 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.865804 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.865811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.865816 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.866049 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.866341 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.866343 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.866345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.866419 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.866858 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.866863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.866970 (+ 107us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.867149 (+ 179us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.867963 (+ 814us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":187,"threads_started":1}
I20260805 23:15:49.869257 31088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.124@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
I20260805 23:15:49.878202 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.869791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58906)
0805 23:15:49.869921 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.869934 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.869999 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.870493 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.870496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.870680 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.870777 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.870782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.871774 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.871779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.872318 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.872326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.872798 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.872802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.872904 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.873443 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.873455 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.874701 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.876630 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.876634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.876636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.876844 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.877136 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.877140 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.877142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.877229 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.877661 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.877664 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.877765 (+ 101us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.877873 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.878022 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 23:15:49.878203 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.869938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.23.88.125:40405)
0805 23:15:49.870247 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.870251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.870265 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:49.870309 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.870312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.870356 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.870480 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.870905 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.871659 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.872443 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.872677 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.872786 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.874871 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.874875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.874877 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.874901 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.876493 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.876951 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.876955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.876957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.877040 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.877341 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.877345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.877346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.877541 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.877661 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.877928 (+ 267us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.878050 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":121,"threads_started":1}
I20260805 23:15:49.879199 31088 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971749, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.124@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
I20260805 23:15:49.887867 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.879724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.0.0.1:38060)
0805 23:15:49.879849 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.879865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.879944 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.880176 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.880178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.880359 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.880437 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.880442 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.881554 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.881557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.882052 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.882058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.882582 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.882584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.882648 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.883180 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.883191 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.884637 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.886393 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.886395 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.886396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.886582 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.886826 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.886828 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.886829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.886899 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.887321 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.887324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.887400 (+ 76us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.887553 (+ 153us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.887714 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 23:15:49.887897 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.879779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.23.88.124:42685)
0805 23:15:49.879899 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.879902 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.879914 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:15:49.880016 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.880018 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.880054 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.880123 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.880542 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.881420 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.882223 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.882481 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.882565 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.884782 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.884786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.884788 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.884809 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.886299 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.886663 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.886665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.886667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.886737 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.886972 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.886974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.886975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.887235 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.887315 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.887632 (+ 317us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.887771 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:15:49.891151 31088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Bootstrap starting.
I20260805 23:15:49.893464 31088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:49.894224 31088 log.cc:824] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:49.896287 31088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: No bootstrap required, opened a new log
I20260805 23:15:49.899662 31088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } }
I20260805 23:15:49.899881 31088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:49.899927 31088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 997df6ed0f6d456692e7c05692f5e658, State: Initialized, Role: FOLLOWER
I20260805 23:15:49.900426 31088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } }
I20260805 23:15:49.901081 31095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } } }
I20260805 23:15:49.901237 31095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:49.901926 31088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:15:49.907775 31106 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:49.907882 31106 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:49.911639 30928 master_runner.cc:428] Detected that this master 997df6ed0f6d456692e7c05692f5e658 is joining an existing cluster
I20260805 23:15:49.911711 30928 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.88.124:42685
I20260805 23:15:49.920811 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30399
I20260805 23:15:49.932763 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:45877
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:46333
--webserver_port=35267
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:49.945201 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.913961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34658)
0805 23:15:49.914309 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.914324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.914438 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.914915 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.914918 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.915252 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.915411 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.915425 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.916254 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.916258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.917158 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.917166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.917888 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.917895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.917992 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.918572 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.918587 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.940085 (+ 21498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.943052 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.943055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.943056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.943355 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.943741 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.943747 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.943752 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.943853 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.944431 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.944438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.944592 (+ 154us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.944783 (+ 191us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.945000 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":120,"threads_started":1}
I20260805 23:15:49.945324 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.914127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.23.88.126:46333)
0805 23:15:49.914280 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.914287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.914415 (+ 128us) server_negotiation.cc:407] Connection header received
0805 23:15:49.914535 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.914538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.914602 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.914690 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.915546 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.916127 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.917297 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.917643 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.917716 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.940322 (+ 22606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.940327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.940330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.940367 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.942858 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.943510 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.943514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.943516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.943616 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.943983 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.943989 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.943994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.944279 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.944380 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.945044 (+ 664us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.945166 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:49.947072 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.915276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38072 (local address 127.23.88.124:42685)
0805 23:15:49.915366 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.915372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.915385 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:15:49.915419 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.915421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.915461 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.915535 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.916101 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.916902 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.917956 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.918266 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.918324 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.940898 (+ 22574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.940903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.940907 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.940934 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.942845 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.945860 (+ 3015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.945864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.945866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.945947 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.946213 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.946215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.946217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.946410 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.946487 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.946807 (+ 320us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.946924 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":41}
I20260805 23:15:49.947799 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.914449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.0.0.1:38072)
0805 23:15:49.914769 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.914783 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.914880 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.915629 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.915632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.915852 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.915976 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.915983 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.917009 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.917012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.917849 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.917858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.918474 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.918477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.918581 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.918977 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.918987 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.940757 (+ 21770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.942987 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.942991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.942996 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.945752 (+ 2756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.946048 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.946050 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.946052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.946132 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.946541 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.946543 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.946626 (+ 83us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.947498 (+ 872us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.947632 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":119,"threads_started":1}
I20260805 23:15:49.952334 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.915469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.23.88.125:40405)
0805 23:15:49.915601 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.915605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.916173 (+ 568us) server_negotiation.cc:407] Connection header received
0805 23:15:49.916345 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.916348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.916401 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.916455 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.917385 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.918217 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.919508 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.919875 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.919989 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.940538 (+ 20549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.940544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.940547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.940582 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.944733 (+ 4151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.950195 (+ 5462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.950203 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.950204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.950327 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.951046 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.951053 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.951056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.951389 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.951484 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.952034 (+ 550us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.952180 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:49.952913 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.914925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58910)
0805 23:15:49.916103 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.916114 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.916240 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.916592 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.916595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.916785 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.917236 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.917246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.918368 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.918390 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.919323 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.919334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.919999 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.920002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.920099 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.920589 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.920605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.940360 (+ 19755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.944868 (+ 4508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.944870 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.944872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.949718 (+ 4846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.950801 (+ 1083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.950803 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.950806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.950903 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.951742 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.951745 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.951827 (+ 82us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.952549 (+ 722us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.952700 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":26,"thread_start_us":116,"threads_started":1}
Aug 05 23:15:49 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971748, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.126@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
I20260805 23:15:49.959818 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.950689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38076 (local address 127.23.88.124:42685)
0805 23:15:49.950810 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.950815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.950826 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:15:49.950890 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.950893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.950951 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.951612 (+ 661us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:49.951725 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.952767 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.954094 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.954295 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.954409 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.956489 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:49.956492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:49.956495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:49.956519 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:49.958026 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.958455 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.958459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.958461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.958540 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:49.958824 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:49.958828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:49.958830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:49.959017 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:49.959169 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.959549 (+ 380us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.959654 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:49.959818 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.950282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.0.0.1:38076)
0805 23:15:49.950495 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.950509 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.950579 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.951161 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.951164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.951358 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:49.951499 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.951510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.952899 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.952905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.953720 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.953734 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.954416 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.954419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.954516 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.955025 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:49.955047 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:49.956346 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:49.958142 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.958145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.958148 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.958355 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.958635 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:49.958639 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:49.958641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:49.958733 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:49.959275 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:49.959277 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:49.959363 (+ 86us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.959492 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.959656 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134}
I20260805 23:15:49.961181 30333 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.88.124:42685: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "997df6ed0f6d456692e7c05692f5e658"
member_type: NON_VOTER
last_known_addr {
host: "127.23.88.124"
port: 42685
}
attrs {
promote: true
}
}
dest_uuid: "cda1e5ea711f4b7a879662344e5df31d"
cas_config_opid_index: 5
I20260805 23:15:49.961776 30333 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } }
I20260805 23:15:49.962942 30923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.964354 30923 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.88.124:42685
I20260805 23:15:49.964545 30923 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.964664 30923 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:49.965358 30928 master.cc:537] Master@127.23.88.124:42685 shutting down...
I20260805 23:15:49.965642 31062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:49.967293 31062 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 23:15:49.969215 30924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:49.971361 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.964858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58922 (local address 127.23.88.125:40405)
0805 23:15:49.965022 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:49.965028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:49.966638 (+ 1610us) server_negotiation.cc:407] Connection header received
0805 23:15:49.966706 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:49.966711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:49.966781 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:49.966874 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:49.967335 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.968099 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.969662 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.970285 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.970388 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.970565 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:49.971150 (+ 585us) server_negotiation.cc:299] Negotiation successful
0805 23:15:49.971234 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260805 23:15:49.971478 30317 consensus_peers.cc:487] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d -> Peer 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124:42685): Couldn't send request to peer 997df6ed0f6d456692e7c05692f5e658. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:15:49.971626 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:49.963296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:58922)
0805 23:15:49.966411 (+ 3115us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:49.966428 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:49.966544 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:49.967043 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:49.967049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:49.967074 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:49.967195 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.967201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.968316 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.968320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.969510 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:49.969521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:49.970635 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:49.970642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:49.970749 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:49.970804 (+ 55us) client_negotiation.cc:769] Sending connection context
0805 23:15:49.970881 (+ 77us) client_negotiation.cc:240] Negotiation successful
0805 23:15:49.970983 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3021}
I20260805 23:15:49.972136 30975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 LEARNER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260805 23:15:49.972615 30924 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260805 23:15:49.974604 30975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.976550 31117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.976696 31117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: LEARNER
I20260805 23:15:49.977062 31004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader cda1e5ea711f4b7a879662344e5df31d. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: NON_VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: true } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.977192 31004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: LEARNER
I20260805 23:15:49.978283 30923 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: attempting to promote NON_VOTER 475ff05cb07a4d92a7b4c17b4dc0eedb to VOTER
I20260805 23:15:49.978761 30923 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } }
I20260805 23:15:49.980197 30975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 LEARNER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260805 23:15:49.980597 30923 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260805 23:15:49.981179 30317 consensus_peers.cc:487] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d -> Peer 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124:42685): Couldn't send request to peer 997df6ed0f6d456692e7c05692f5e658. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:15:49.982277 30923 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.982873 30928 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:49.983171 30928 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:49.983273 30928 tablet_replica.cc:330] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: stopping tablet replica
I20260805 23:15:49.984522 30924 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.984635 30924 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:49.985491 30924 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.985595 30924 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:49.986436 30975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.988665 31004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:49.988818 31004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:50.000614 30928 master.cc:552] Master@127.23.88.124:42685 shutdown complete.
I20260805 23:15:50.000697 30928 master_runner.cc:305] Clearing existing system tablet
I20260805 23:15:50.000946 30928 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:50.001051 30928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:50.002399 30928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260805 23:15:50.003408 31121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.003646 30928 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:50.003783 30928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:50.003819 30928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "997df6ed0f6d456692e7c05692f5e658"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "aee1e30a2ca3b6eb09a0a3efe9ad3f8d"
server_key_iv: "aa919cbc4f072b991bbbfcadc0be4fe6"
server_key_version: "encryptionkey@0"
I20260805 23:15:50.005707 30928 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:50.007829 30928 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260805 23:15:50.007951 30928 log.cc:1196] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260805 23:15:50.008231 30928 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Deleting consensus metadata
I20260805 23:15:50.008437 30928 master_runner.cc:315] Copying system tablet from 127.23.88.126:46333
I20260805 23:15:50.009624 30928 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: tablet copy: Beginning tablet copy session from remote peer at address 127.23.88.126:46333
I20260805 23:15:50.018275 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.010292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34664 (local address 127.23.88.126:46333)
0805 23:15:50.010435 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.010440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.010664 (+ 224us) server_negotiation.cc:407] Connection header received
0805 23:15:50.010815 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.010818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.010872 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.010960 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.011528 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.012298 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.013383 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.013628 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.013744 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.014945 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.014949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.014951 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.014979 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.016577 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.017021 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.017026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.017028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.017116 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.017401 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.017404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.017406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.017602 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.017709 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.017974 (+ 265us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.018086 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:50.018914 31127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.010204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:34664)
0805 23:15:50.010566 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.010578 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.010715 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.010985 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.010988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.011228 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.011344 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.011350 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.012581 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.012583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.013259 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.013267 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.013750 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.013753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.013852 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.014452 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.014466 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.014796 (+ 330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.016695 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.016700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.016703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.016911 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.017210 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.017214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.017216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.017306 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.017701 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.017704 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.017808 (+ 104us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.017919 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.018081 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":91,"threads_started":1}
I20260805 23:15:50.019409 30353 tablet_copy_service.cc:140] P cda1e5ea711f4b7a879662344e5df31d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 997df6ed0f6d456692e7c05692f5e658 ({username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:34664)
I20260805 23:15:50.019528 30353 tablet_copy_service.cc:161] P cda1e5ea711f4b7a879662344e5df31d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 997df6ed0f6d456692e7c05692f5e658 at {username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:34664: session id = 997df6ed0f6d456692e7c05692f5e658-00000000000000000000000000000000
I20260805 23:15:50.020411 30353 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:15:50.021695 30928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260805 23:15:50.023808 30928 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: tablet copy: Starting download of 0 data blocks...
I20260805 23:15:50.024029 30928 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: tablet copy: Starting download of 1 WAL segments...
I20260805 23:15:50.025646 30928 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:15:50.027316 30353 tablet_copy_service.cc:342] P cda1e5ea711f4b7a879662344e5df31d: Request end of tablet copy session 997df6ed0f6d456692e7c05692f5e658-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:34664
I20260805 23:15:50.027417 30353 tablet_copy_service.cc:433] P cda1e5ea711f4b7a879662344e5df31d: ending tablet copy session 997df6ed0f6d456692e7c05692f5e658-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 997df6ed0f6d456692e7c05692f5e658
I20260805 23:15:50.034288 30928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:50.036235 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.036334 31131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.036680 31129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:50.041188 30928 server_base.cc:1034] running on GCE node
I20260805 23:15:50.041389 30928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:50.041771 30928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:50.042973 30928 hybrid_clock.cc:648] HybridClock initialized: now 1785971750042959 us; error 32 us; skew 500 ppm
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:50.044697 30928 init.cc:377] Logged in from keytab as oryx/127.23.88.124@KRBTEST.COM (short username oryx)
I20260805 23:15:50.045264 30928 webserver.cc:460] Webserver started at http://127.23.88.124:41595/ using document root <none> and password file <none>
I20260805 23:15:50.045490 30928 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:50.045545 30928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:50.046741 30928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:50.047869 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.048127 30928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:50.048225 30928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "997df6ed0f6d456692e7c05692f5e658"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "aee1e30a2ca3b6eb09a0a3efe9ad3f8d"
server_key_iv: "aa919cbc4f072b991bbbfcadc0be4fe6"
server_key_version: "encryptionkey@0"
I20260805 23:15:50.048338 30928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:50.060848 30674 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
I20260805 23:15:50.090114 30928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:50.092350 30928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:50.097483 30928 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.124:42685
I20260805 23:15:50.097501 31196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.124:42685 every 8 connection(s)
I20260805 23:15:50.098083 30928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
W20260805 23:15:50.098163 31115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:50.098520 31115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:50.098603 31115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260805 23:15:50.100203 31197 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:15:50.101140 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Bootstrap starting.
W20260805 23:15:50.104873 31115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:50.104950 31115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:50.105048 31115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:50.110194 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:15:50.110532 31197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Bootstrap complete.
I20260805 23:15:50.111065 31197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } }
I20260805 23:15:50.111276 31197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 997df6ed0f6d456692e7c05692f5e658, State: Initialized, Role: LEARNER
I20260805 23:15:50.111404 31197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } }
I20260805 23:15:50.111812 31200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:50.111989 31197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:50.112025 31200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:15:50.113660 31115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:45877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=35267
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:50.114959 31115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:50.115247 31212 catalog_manager.cc:1259] Loaded cluster ID: 84c7ee6b29114756b0299c5d6d2c4439
I20260805 23:15:50.115386 31212 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: loading cluster ID for follower catalog manager: success
I20260805 23:15:50.116247 31115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:50.117401 31212 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: acquiring CA information for follower catalog manager: success
I20260805 23:15:50.118479 31212 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:15:50.128311 31115 server_base.cc:1034] running on GCE node
W20260805 23:15:50.128258 31217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.128271 31214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.128284 31215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:50.128997 31115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:50.129644 31115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:50.130838 31115 hybrid_clock.cc:648] HybridClock initialized: now 1785971750130808 us; error 46 us; skew 500 ppm
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:50.133739 31115 init.cc:377] Logged in from keytab as oryx/127.23.88.65@KRBTEST.COM (short username oryx)
I20260805 23:15:50.134716 31115 webserver.cc:460] Webserver started at http://127.23.88.65:35267/ using document root <none> and password file <none>
I20260805 23:15:50.135365 31115 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:50.135494 31115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:50.139688 31115 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:15:50.141840 31224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.143329 31115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:50.143471 31115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ccbd5600c41f4c7296ad486a288199c4"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "fb1bc1c2ed905d34458baa868005e674"
server_key_iv: "83013a69ddf2edccd571f11a85f9b59a"
server_key_version: "encryptionkey@0"
I20260805 23:15:50.144049 31115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:50.168339 31115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:50.171451 31115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:50.171697 31115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:50.172400 31115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:50.173321 31115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:50.173395 31115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.173467 31115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:50.173520 31115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.183331 31115 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:45877
I20260805 23:15:50.183350 31341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:45877 every 8 connection(s)
I20260805 23:15:50.184427 31115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:50.194388 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 31115
I20260805 23:15:50.194648 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30540
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
I20260805 23:15:50.203266 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.186754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:42037 (local address 127.23.88.125:40405)
0805 23:15:50.186883 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.186888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.188046 (+ 1158us) server_negotiation.cc:407] Connection header received
0805 23:15:50.189411 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.189416 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.189486 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.189570 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.191259 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.191800 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194189 (+ 2389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.194424 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194545 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.198084 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.198088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.198091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.198125 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.200183 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.200879 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.200886 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.200890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.200980 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.201449 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.201455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.201458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.201798 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.201912 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.202729 (+ 817us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.202906 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 23:15:50.204319 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.187155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.23.88.65:42037)
0805 23:15:50.187905 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.187949 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.189118 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.189810 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.189820 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.190429 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.191025 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.191039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.191972 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.191976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.193520 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.193526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.194559 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.194657 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.195288 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.195303 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.197842 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.200331 (+ 2489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.200339 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.200360 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.200752 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.201105 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.201111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.201116 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.201311 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.201960 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.201968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.202367 (+ 399us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.202620 (+ 253us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.203105 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":240,"spinlock_wait_cycles":11776,"thread_start_us":261,"threads_started":1}
I20260805 23:15:50.205180 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.188524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.23.88.65:49659)
0805 23:15:50.188814 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.188831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.189150 (+ 319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.189885 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.189888 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.190423 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.191021 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.191037 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.192435 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.192439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.193403 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.193413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.194111 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194117 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.194332 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.195125 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.195153 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.200546 (+ 5393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.202794 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.202797 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.202799 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.203145 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.203748 (+ 603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.203751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.203753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.203843 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.204305 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.204310 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.204750 (+ 440us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.204869 (+ 119us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.205021 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":48,"spinlock_wait_cycles":2432,"thread_start_us":99,"threads_started":1}
I20260805 23:15:50.205813 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.188671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:49659 (local address 127.23.88.124:42685)
0805 23:15:50.188927 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.188943 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.188960 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:15:50.189687 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.189690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.189738 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.189840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.191250 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.192318 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.193539 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.193899 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194017 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.200784 (+ 6767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.200792 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.200799 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.200844 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.202672 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.203264 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.203267 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.203269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.203635 (+ 366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.203949 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.203952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.203954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.204169 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.204258 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.205545 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.205661 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":89,"threads_started":1}
I20260805 23:15:50.205796 31344 heartbeater.cc:347] Connected to a master server at 127.23.88.125:40405
I20260805 23:15:50.206305 31344 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.206837 31344 heartbeater.cc:511] Master 127.23.88.125:40405 requested a full tablet report, sending...
I20260805 23:15:50.207572 31342 heartbeater.cc:347] Connected to a master server at 127.23.88.124:42685
I20260805 23:15:50.207660 31342 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.207839 31342 heartbeater.cc:511] Master 127.23.88.124:42685 requested a full tablet report, sending...
I20260805 23:15:50.209547 31158 ts_manager.cc:194] Registered new tserver with Master: ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877)
I20260805 23:15:50.210598 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.186590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:58261 (local address 127.23.88.126:46333)
0805 23:15:50.186741 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.186745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.188051 (+ 1306us) server_negotiation.cc:407] Connection header received
0805 23:15:50.189385 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.189389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.189479 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.189528 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.191233 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.192078 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.193629 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.193968 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.194064 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.198083 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.198088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.198091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.198126 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.208644 (+ 10518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.209165 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.209169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.209171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.209272 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.209592 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.209596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.209598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.209829 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.209949 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.210241 (+ 292us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.210380 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:15:50.211042 30965 ts_manager.cc:194] Registered new tserver with Master: ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877)
I20260805 23:15:50.211133 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:46667
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:46333
--webserver_port=35169
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:50.211532 31347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.187155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:58261)
0805 23:15:50.187859 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.187929 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.189118 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.189810 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.189820 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.190429 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.191025 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.191039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.192201 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.192204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.193509 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.193521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.195257 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.195261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.195374 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.195861 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.195877 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.197842 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.208782 (+ 10940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.208785 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.208787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.209029 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.209380 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.209383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.209385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.209485 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.209935 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.209938 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.210037 (+ 99us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.210759 (+ 722us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.210925 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":262,"spinlock_wait_cycles":102912,"thread_start_us":237,"threads_started":1}
I20260805 23:15:50.213367 31343 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:50.213469 31343 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.213693 31343 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
I20260805 23:15:50.214525 30333 ts_manager.cc:194] Re-registered known tserver with Master: ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877)
I20260805 23:15:50.215257 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:58261
I20260805 23:15:50.295720 30816 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
W20260805 23:15:50.331594 31351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:50.331974 31351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:50.332055 31351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:50.340054 31351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:50.340142 31351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:50.340211 31351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:50.350432 31351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:46667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=35169
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:50.352142 31351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:50.353732 31351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:50.367799 31356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:50.368611 31351 server_base.cc:1034] running on GCE node
W20260805 23:15:50.368881 31361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.375866 31357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:50.376657 31351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:50.377529 31351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:50.378798 31351 hybrid_clock.cc:648] HybridClock initialized: now 1785971750378784 us; error 65 us; skew 500 ppm
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:50.382452 31351 init.cc:377] Logged in from keytab as oryx/127.23.88.66@KRBTEST.COM (short username oryx)
I20260805 23:15:50.383811 31351 webserver.cc:460] Webserver started at http://127.23.88.66:35169/ using document root <none> and password file <none>
I20260805 23:15:50.384487 31351 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:50.384596 31351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:50.389081 31351 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260805 23:15:50.391454 31368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.392890 31351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:15:50.393050 31351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f"
format_stamp: "Formatted at 2026-08-05 23:15:48 on dist-test-slave-vkjp"
server_key: "0454c331efec2f63f0ca8cad84f61c1f"
server_key_iv: "a2914770d03d5bdbaf6db0bcc1f79341"
server_key_version: "encryptionkey@0"
I20260805 23:15:50.393702 31351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:50.420395 31351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:50.424088 31351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:50.424314 31351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:50.424963 31351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:50.425899 31351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:50.425973 31351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.426047 31351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:50.426098 31351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.435912 31351 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:46667
I20260805 23:15:50.435945 31487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:46667 every 8 connection(s)
I20260805 23:15:50.436977 31351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:50.440019 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 31351
I20260805 23:15:50.440243 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30679
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
I20260805 23:15:50.451737 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:35477
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:46333
--webserver_port=44503
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:34183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:50.456455 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.439216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:33359 (local address 127.23.88.124:42685)
0805 23:15:50.439398 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.439403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.442520 (+ 3117us) server_negotiation.cc:407] Connection header received
0805 23:15:50.442730 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.442734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.442790 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.442842 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.444396 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.445252 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.447710 (+ 2458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.448059 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.448155 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.451539 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.451543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.451546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.451578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.453713 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.454454 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.454459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.454463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.454559 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.454966 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.454969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.454971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.455234 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.455344 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.456028 (+ 684us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.456183 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260805 23:15:50.457649 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.445956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.0.0.1:45648)
0805 23:15:50.446200 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.446213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.446285 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.451633 (+ 5348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.451640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.451668 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.451773 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.451780 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.454533 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.454538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.455985 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.455999 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.457143 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.457147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.457263 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.457323 (+ 60us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.457379 (+ 56us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.457482 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260805 23:15:50.458500 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.446431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45648 (local address 127.23.88.124:42685)
0805 23:15:50.451306 (+ 4875us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.451311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.451329 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:15:50.451380 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.451383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.451456 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.451922 (+ 466us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.452056 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.454363 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.456165 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.456985 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.458006 (+ 1021us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.458204 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.458249 (+ 45us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.458353 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4777,"thread_start_us":93,"threads_started":1}
I20260805 23:15:50.459208 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.439551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.23.88.66:33359)
0805 23:15:50.442429 (+ 2878us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.442444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.442592 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.443050 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.443052 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.444146 (+ 1094us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.444258 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.444264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.446649 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.446652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.447538 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.447552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.448334 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.448339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.448458 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.449055 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.449070 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.451308 (+ 2238us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.453868 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.453875 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.453896 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.454303 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.454677 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.454683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.454688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.454851 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.455345 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.455354 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.455727 (+ 373us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.457056 (+ 1329us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.457494 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2784,"mutex_wait_us":25,"thread_start_us":156,"threads_started":1}
I20260805 23:15:50.459151 31168 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260805 23:15:50.459760 31200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cda1e5ea711f4b7a879662344e5df31d. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:50.459951 31200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: LEARNER
I20260805 23:15:50.460665 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.439550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.23.88.66:53353)
0805 23:15:50.440054 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.440109 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.442001 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.442852 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.442861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.443514 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.444204 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.444226 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.445429 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.445432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.446809 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.446817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.447672 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.447679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.447890 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.448731 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.448761 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.452084 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.456966 (+ 4882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.456970 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.456972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.457245 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.458081 (+ 836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.458084 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.458086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.458183 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.460076 (+ 1893us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.460079 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.460187 (+ 108us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.460308 (+ 121us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.460481 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":139,"threads_started":1}
I20260805 23:15:50.460930 31488 heartbeater.cc:347] Connected to a master server at 127.23.88.124:42685
I20260805 23:15:50.461247 31488 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.461382 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.441592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:39215 (local address 127.23.88.126:46333)
0805 23:15:50.441737 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.441742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.445216 (+ 3474us) server_negotiation.cc:407] Connection header received
0805 23:15:50.445442 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.445450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.445518 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.445581 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.446354 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.447400 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.449007 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.449505 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.449606 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.452651 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.452658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.452665 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.452702 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.456357 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.457356 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.457363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.457366 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.457463 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.458293 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.458297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.458299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.458527 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.458641 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.461086 (+ 2445us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.461215 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:15:50.461895 31488 heartbeater.cc:511] Master 127.23.88.124:42685 requested a full tablet report, sending...
I20260805 23:15:50.462001 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.438946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:53353 (local address 127.23.88.125:40405)
0805 23:15:50.439230 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.439236 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.440217 (+ 981us) server_negotiation.cc:407] Connection header received
0805 23:15:50.442270 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.442277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.442347 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.442419 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.444403 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.445217 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.446945 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.447331 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.447417 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.452715 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.452720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.452724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.452767 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.455405 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.457843 (+ 2438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.457847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.457849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.457960 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.459276 (+ 1316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.459280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.459282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.459546 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.459654 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.461672 (+ 2018us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.461828 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260805 23:15:50.462553 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.440664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:39215)
0805 23:15:50.445109 (+ 4445us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.445127 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.445305 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.445762 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.445768 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.446029 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.446194 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.446205 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.447563 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.447570 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.448845 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.448861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.449865 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.449872 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.450018 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.450640 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.450655 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.452511 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.456487 (+ 3976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.456490 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.456492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.456756 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.457576 (+ 820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.457579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.457582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.457682 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.458694 (+ 1012us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.458697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.458820 (+ 123us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.458940 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.459127 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4346,"thread_start_us":367,"threads_started":1}
I20260805 23:15:50.462720 31490 heartbeater.cc:347] Connected to a master server at 127.23.88.125:40405
I20260805 23:15:50.462805 31490 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.463009 31490 heartbeater.cc:511] Master 127.23.88.125:40405 requested a full tablet report, sending...
I20260805 23:15:50.463222 31158 ts_manager.cc:194] Registered new tserver with Master: e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:50.463606 31489 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:50.463688 31489 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.463843 30965 ts_manager.cc:194] Registered new tserver with Master: e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:50.463868 31489 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
I20260805 23:15:50.464500 30333 ts_manager.cc:194] Re-registered known tserver with Master: e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:50.465142 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:39215
W20260805 23:15:50.571632 31496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:50.571938 31496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:50.572033 31496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:50.576231 31496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:50.576327 31496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:50.576416 31496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:50.581998 31496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:34183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:35477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=44503
--tserver_master_addrs=127.23.88.126:46333,127.23.88.125:40405,127.23.88.124:42685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:50.583238 31496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:50.584439 31496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:50.594920 31505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.594937 31503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:50.594937 31502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:50.595306 31496 server_base.cc:1034] running on GCE node
I20260805 23:15:50.595676 31496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:50.596329 31496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:50.597496 31496 hybrid_clock.cc:648] HybridClock initialized: now 1785971750597477 us; error 28 us; skew 500 ppm
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:50.600757 31496 init.cc:377] Logged in from keytab as oryx/127.23.88.67@KRBTEST.COM (short username oryx)
I20260805 23:15:50.601693 31496 webserver.cc:460] Webserver started at http://127.23.88.67:44503/ using document root <none> and password file <none>
I20260805 23:15:50.602291 31496 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:50.602402 31496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:50.606534 31496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:50.608809 31512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.610147 31496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:15:50.610286 31496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "140d6d27f0214b86a6211f44f74aa97e"
format_stamp: "Formatted at 2026-08-05 23:15:49 on dist-test-slave-vkjp"
server_key: "4566ca225e0105376a2dc6e1a42a5a54"
server_key_iv: "6f2f9363fb505e1d4ab61e2dcf4856fe"
server_key_version: "encryptionkey@0"
I20260805 23:15:50.610845 31496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:50.635412 31496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:50.639071 31496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:50.639310 31496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:50.639997 31496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:50.640971 31496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:50.641060 31496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.641129 31496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:50.641180 31496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:50.650619 31496 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35477
I20260805 23:15:50.650673 31629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35477 every 8 connection(s)
I20260805 23:15:50.651794 31496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:50.660841 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 31496
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.124@KRBTEST.COM
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.125@KRBTEST.COM
Aug 05 23:15:50 dist-test-slave-vkjp krb5kdc[30286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971750, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:50.674754 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.655752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:50675 (local address 127.23.88.124:42685)
0805 23:15:50.655878 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.655883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.655899 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:50.656131 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.656135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.656190 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.656244 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.657761 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.658302 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660111 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660439 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660546 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.665175 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.665179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.665182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.665210 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.670468 (+ 5258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672554 (+ 2086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.672557 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.672559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.672651 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.673577 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.673585 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.673590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.673841 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.673944 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.674398 (+ 454us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.674566 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:15:50.675586 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.654264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.23.88.67:40447)
0805 23:15:50.654760 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.654813 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.655896 (+ 1083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.656460 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.656468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.657068 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.657615 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.657623 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.658462 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.658465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.659987 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.659998 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660915 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.661049 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.661752 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.661766 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.664959 (+ 3193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.671542 (+ 6583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.671546 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.671549 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.671848 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.672550 (+ 702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.672555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.672659 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.673246 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.673257 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.673664 (+ 407us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.673896 (+ 232us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.674333 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":256,"spinlock_wait_cycles":28032,"thread_start_us":161,"threads_started":1}
I20260805 23:15:50.675832 31636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.655363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.23.88.67:50675)
0805 23:15:50.655648 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.655666 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.655899 (+ 233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.656460 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.656468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.657049 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.657614 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.657623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.658462 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.658465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.659998 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.660005 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660566 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660572 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.660707 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.661414 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.661441 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.664960 (+ 3519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.672168 (+ 7208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672173 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.672175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.672442 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.673326 (+ 884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.673332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.673336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.673460 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.673995 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.674001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.674097 (+ 96us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.674197 (+ 100us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.675206 (+ 1009us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":1048,"spinlock_wait_cycles":37760,"thread_start_us":99,"threads_started":1}
I20260805 23:15:50.676473 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.654264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:59851)
0805 23:15:50.654760 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.654813 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.655888 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.656459 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.656466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.657047 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:50.657597 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.657611 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.659056 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.659063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.659776 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.659785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660357 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.661097 (+ 737us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.661656 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:50.661667 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:50.664960 (+ 3293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:50.670630 (+ 5670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.670639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.670662 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.671139 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.671807 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:50.671813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:50.671817 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:50.672056 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:50.673247 (+ 1191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:50.673257 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:50.676030 (+ 2773us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.676124 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.676272 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":2651,"spinlock_wait_cycles":6016,"thread_start_us":128,"threads_started":1}
I20260805 23:15:50.676759 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.663949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.23.88.125:40405)
0805 23:15:50.665959 (+ 2010us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.665963 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.665979 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:50.666159 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.666165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.666220 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.666332 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.667399 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.668295 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.669588 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.669961 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.670053 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.672557 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.672561 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.672565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.672591 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.674401 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.674978 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.674983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.674986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.675073 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.675424 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.675427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.675429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.675641 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.675805 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.676437 (+ 632us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.676577 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1919,"thread_start_us":104,"threads_started":1}
I20260805 23:15:50.676993 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.653765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:40447 (local address 127.23.88.125:40405)
0805 23:15:50.653966 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.653971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.654917 (+ 946us) server_negotiation.cc:407] Connection header received
0805 23:15:50.656131 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.656135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.656190 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.656244 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.657774 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.658303 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660486 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660784 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660900 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.665168 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.665175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.665181 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.665216 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.671411 (+ 6195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672320 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.672326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.672330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.672426 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672782 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.672787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.672788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.673072 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.673182 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.676701 (+ 3519us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.676824 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260805 23:15:50.677237 31630 heartbeater.cc:347] Connected to a master server at 127.23.88.124:42685
I20260805 23:15:50.677500 31630 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.677649 31632 heartbeater.cc:347] Connected to a master server at 127.23.88.125:40405
I20260805 23:15:50.677721 31632 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.677757 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.653810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:59851 (local address 127.23.88.126:46333)
0805 23:15:50.654062 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.654067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.654922 (+ 855us) server_negotiation.cc:407] Connection header received
0805 23:15:50.656112 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.656115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.656180 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.656284 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.658436 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.658937 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.659919 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.660213 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.660315 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.665175 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.665179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.665181 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.665210 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.668958 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.671562 (+ 2604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.671565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.671567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.671684 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.672777 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.672782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.672786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.673073 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.673178 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.677422 (+ 4244us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.677581 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"spinlock_wait_cycles":9472}
I20260805 23:15:50.678133 31630 heartbeater.cc:511] Master 127.23.88.124:42685 requested a full tablet report, sending...
I20260805 23:15:50.678273 31632 heartbeater.cc:511] Master 127.23.88.125:40405 requested a full tablet report, sending...
I20260805 23:15:50.678992 31631 heartbeater.cc:347] Connected to a master server at 127.23.88.126:46333
I20260805 23:15:50.679068 31631 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:50.679253 31631 heartbeater.cc:511] Master 127.23.88.126:46333 requested a full tablet report, sending...
I20260805 23:15:50.679298 31158 ts_manager.cc:194] Registered new tserver with Master: 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:50.679382 30964 ts_manager.cc:194] Registered new tserver with Master: 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:50.679622 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.664444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45664 (local address 127.23.88.124:42685)
0805 23:15:50.664580 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.664585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.666277 (+ 1692us) server_negotiation.cc:407] Connection header received
0805 23:15:50.666445 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.666448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.666492 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.666542 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.667488 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.668276 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.669438 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.669768 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.669869 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.672012 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.672018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.672023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.672054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.675056 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.677957 (+ 2901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.677963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.677966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.678066 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.678395 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.678399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.678400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.678620 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.678746 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.679169 (+ 423us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.679292 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:15:50.679966 30333 ts_manager.cc:194] Re-registered known tserver with Master: 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:50.680546 30333 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:59851
I20260805 23:15:50.681617 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.663838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.23.88.126:46333)
0805 23:15:50.664102 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.664107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.664730 (+ 623us) server_negotiation.cc:407] Connection header received
0805 23:15:50.664953 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.664956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.665011 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.665075 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.666259 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.667167 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.668436 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.668794 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.668875 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.671232 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.671236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.671239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.671275 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.673698 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.675433 (+ 1735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.675436 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.675438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.675532 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.680639 (+ 5107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.680644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.680646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.680912 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.681079 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.681335 (+ 256us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.681463 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"spinlock_wait_cycles":10624,"thread_start_us":73,"threads_started":1}
I20260805 23:15:50.684365 30333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:50.687785 30333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:50.690654 31119 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: NON_VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: true } } }
I20260805 23:15:50.690829 31119 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:50.694581 31119 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: attempting to promote NON_VOTER 997df6ed0f6d456692e7c05692f5e658 to VOTER
I20260805 23:15:50.694976 31119 consensus_queue.cc:237] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } }
I20260805 23:15:50.696298 30975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 23:15:50.696728 31650 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260805 23:15:50.698267 31119 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.698804 31168 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 LEARNER]: Refusing update from remote peer cda1e5ea711f4b7a879662344e5df31d: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260805 23:15:50.698786 30975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.699414 31648 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [LEADER]: Connected to new peer: Peer: permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260805 23:15:50.700474 31168 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.700944 31648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.701116 31648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:50.701311 31648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.701421 31648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:50.702205 31658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.702360 31658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:50.702342 31200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.702461 31200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:50.703375 31648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cda1e5ea711f4b7a879662344e5df31d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:50.703493 31648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:50.710592 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.705025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46667 (local address 127.0.0.1:38604)
0805 23:15:50.705132 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.705151 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.705238 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.706433 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.706436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.706448 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.706566 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.706572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.708000 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.708003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.709297 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.709308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.710199 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.710203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.710287 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.710309 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.710371 (+ 62us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.710445 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":60}
I20260805 23:15:50.710667 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.705376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.23.88.66:46667)
0805 23:15:50.705837 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.705853 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.705880 (+ 27us) server_negotiation.cc:407] Connection header received
0805 23:15:50.705969 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.705974 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.706267 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.706464 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.706694 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.707863 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.709439 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.710070 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.710202 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.710382 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.710436 (+ 54us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.710529 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":220,"threads_started":1}
I20260805 23:15:50.711056 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.705689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45877 (local address 127.0.0.1:33986)
0805 23:15:50.706346 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.706361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.706446 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.707214 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.707217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.707235 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.707334 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.707341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.708552 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.708557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.709356 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.709393 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.710615 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.710620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.710712 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.710748 (+ 36us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.710812 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.710881 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":24,"thread_start_us":130,"threads_started":1}
I20260805 23:15:50.711551 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.706113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.23.88.65:45877)
0805 23:15:50.706753 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.706762 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.706786 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:15:50.706862 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.706870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.707050 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.707238 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.707430 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.708441 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.709491 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.710302 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.710426 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.710553 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.711302 (+ 749us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.711411 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":109,"threads_started":1}
I20260805 23:15:50.712929 31660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.706022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35477 (local address 127.0.0.1:54364)
0805 23:15:50.706715 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.706730 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.706821 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.707975 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.707978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.708000 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.708145 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.708151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.709558 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.709561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.710649 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.710657 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.712475 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.712478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.712571 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.712595 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.712658 (+ 63us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.712787 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":26,"thread_start_us":168,"threads_started":1}
I20260805 23:15:50.712996 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.706784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54364 (local address 127.23.88.67:35477)
0805 23:15:50.707212 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.707223 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.707246 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:15:50.707313 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.707318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.707686 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.707825 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.708345 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.709437 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.710780 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.712351 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.712477 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.712674 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.712724 (+ 50us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.712853 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":98,"threads_started":1}
I20260805 23:15:50.714207 31418 tablet_service.cc:1467] Processing CreateTablet for tablet 76431dc616574b0ebb3989153ed60ac3 (DEFAULT_TABLE table=test-table [id=72fb195139c3434f879915a2c54f5778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:50.714192 31274 tablet_service.cc:1467] Processing CreateTablet for tablet 76431dc616574b0ebb3989153ed60ac3 (DEFAULT_TABLE table=test-table [id=72fb195139c3434f879915a2c54f5778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:50.715440 31418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76431dc616574b0ebb3989153ed60ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:50.715664 31274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76431dc616574b0ebb3989153ed60ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:50.715554 31562 tablet_service.cc:1467] Processing CreateTablet for tablet 76431dc616574b0ebb3989153ed60ac3 (DEFAULT_TABLE table=test-table [id=72fb195139c3434f879915a2c54f5778]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:50.716432 31562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76431dc616574b0ebb3989153ed60ac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:50.725032 31666 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Bootstrap starting.
I20260805 23:15:50.727329 31666 tablet_bootstrap.cc:654] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:50.727556 31665 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Bootstrap starting.
I20260805 23:15:50.727556 31664 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Bootstrap starting.
I20260805 23:15:50.728299 31666 log.cc:824] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:50.729923 31665 tablet_bootstrap.cc:654] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:50.729933 31664 tablet_bootstrap.cc:654] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:50.730753 31665 log.cc:824] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:50.730780 31664 log.cc:824] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:50.732765 31666 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: No bootstrap required, opened a new log
I20260805 23:15:50.732805 31665 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: No bootstrap required, opened a new log
I20260805 23:15:50.732999 31665 ts_tablet_manager.cc:1397] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260805 23:15:50.733001 31666 ts_tablet_manager.cc:1397] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260805 23:15:50.736984 31666 raft_consensus.cc:348] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.737268 31664 tablet_bootstrap.cc:492] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: No bootstrap required, opened a new log
I20260805 23:15:50.737304 31666 raft_consensus.cc:374] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:50.737361 31666 raft_consensus.cc:729] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92980f3ebb6487d8f56a5b9bd2f6e0f, State: Initialized, Role: FOLLOWER
I20260805 23:15:50.737485 31664 ts_tablet_manager.cc:1397] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260805 23:15:50.737536 31665 raft_consensus.cc:348] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.737809 31665 raft_consensus.cc:374] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:50.737861 31665 raft_consensus.cc:729] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140d6d27f0214b86a6211f44f74aa97e, State: Initialized, Role: FOLLOWER
I20260805 23:15:50.737887 31666 consensus_queue.cc:260] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.738349 31665 consensus_queue.cc:260] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.739122 31666 ts_tablet_manager.cc:1428] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260805 23:15:50.739161 31489 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
I20260805 23:15:50.740581 31665 ts_tablet_manager.cc:1428] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260805 23:15:50.740887 31631 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
I20260805 23:15:50.746192 31664 raft_consensus.cc:348] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.746551 31664 raft_consensus.cc:374] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:50.746622 31664 raft_consensus.cc:729] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbd5600c41f4c7296ad486a288199c4, State: Initialized, Role: FOLLOWER
I20260805 23:15:50.747153 31664 consensus_queue.cc:260] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.747923 31343 heartbeater.cc:503] Master 127.23.88.126:46333 was elected leader, sending a full tablet report...
I20260805 23:15:50.748319 31664 ts_tablet_manager.cc:1428] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
I20260805 23:15:50.863494 31670 raft_consensus.cc:482] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:50.863799 31670 raft_consensus.cc:504] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.865044 31670 leader_election.cc:288] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877), 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:50.870249 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.865411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:57805 (local address 127.23.88.67:35477)
0805 23:15:50.865542 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.865547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.865563 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:50.865649 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.865652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.865703 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.865792 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.866186 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.867033 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.868863 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.869489 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.869622 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.869684 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.870032 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.870108 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:15:50.870249 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.865424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:43563 (local address 127.23.88.65:45877)
0805 23:15:50.865569 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.865573 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.865899 (+ 326us) server_negotiation.cc:407] Connection header received
0805 23:15:50.865990 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.865993 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.866032 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.866091 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:50.866356 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.867071 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.868863 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.869495 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.869623 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.869684 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.870032 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.870108 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:15:50.870261 31492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.865301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45877 (local address 127.23.88.66:43563)
0805 23:15:50.865786 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.865803 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.865903 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.866130 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.866136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.866160 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.866251 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.866256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.867202 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.867205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.868720 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.868733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.869633 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.869637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.869810 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.869915 (+ 105us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.870001 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.870094 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"mutex_wait_us":499,"spinlock_wait_cycles":4480}
I20260805 23:15:50.870285 31494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.865301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35477 (local address 127.23.88.66:57805)
0805 23:15:50.865413 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:50.865430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:50.865547 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:50.865862 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:50.865865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:50.865891 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:50.866043 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.866049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.867198 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.867202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.868720 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:50.868733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.869633 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.869637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:50.869820 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.869916 (+ 96us) client_negotiation.cc:769] Sending connection context
0805 23:15:50.870000 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 23:15:50.870094 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":219}
I20260805 23:15:50.871045 31295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76431dc616574b0ebb3989153ed60ac3" candidate_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbd5600c41f4c7296ad486a288199c4" is_pre_election: true
I20260805 23:15:50.871165 31583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76431dc616574b0ebb3989153ed60ac3" candidate_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140d6d27f0214b86a6211f44f74aa97e" is_pre_election: true
I20260805 23:15:50.871464 31295 raft_consensus.cc:2454] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e92980f3ebb6487d8f56a5b9bd2f6e0f in term 0.
I20260805 23:15:50.871522 31583 raft_consensus.cc:2454] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e92980f3ebb6487d8f56a5b9bd2f6e0f in term 0.
I20260805 23:15:50.872015 31370 leader_election.cc:302] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 140d6d27f0214b86a6211f44f74aa97e, ccbd5600c41f4c7296ad486a288199c4, e92980f3ebb6487d8f56a5b9bd2f6e0f; no voters:
I20260805 23:15:50.872337 31670 raft_consensus.cc:2781] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:50.872421 31670 raft_consensus.cc:482] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:50.872452 31670 raft_consensus.cc:3037] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:50.873562 31670 raft_consensus.cc:504] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.873977 31670 leader_election.cc:288] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [CANDIDATE]: Term 1 election: Requested vote from peers ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877), 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477)
I20260805 23:15:50.874379 31583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76431dc616574b0ebb3989153ed60ac3" candidate_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "140d6d27f0214b86a6211f44f74aa97e"
I20260805 23:15:50.874385 31295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76431dc616574b0ebb3989153ed60ac3" candidate_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbd5600c41f4c7296ad486a288199c4"
I20260805 23:15:50.874516 31583 raft_consensus.cc:3037] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:50.874534 31295 raft_consensus.cc:3037] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:50.875677 31583 raft_consensus.cc:2454] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e92980f3ebb6487d8f56a5b9bd2f6e0f in term 1.
I20260805 23:15:50.875689 31295 raft_consensus.cc:2454] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e92980f3ebb6487d8f56a5b9bd2f6e0f in term 1.
I20260805 23:15:50.875993 31372 leader_election.cc:302] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 140d6d27f0214b86a6211f44f74aa97e, ccbd5600c41f4c7296ad486a288199c4, e92980f3ebb6487d8f56a5b9bd2f6e0f; no voters:
I20260805 23:15:50.876217 31670 raft_consensus.cc:2781] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:50.876499 31670 raft_consensus.cc:686] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 LEADER]: Becoming Leader. State: Replica: e92980f3ebb6487d8f56a5b9bd2f6e0f, State: Running, Role: LEADER
I20260805 23:15:50.876876 31670 consensus_queue.cc:237] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } }
I20260805 23:15:50.880779 30332 catalog_manager.cc:5495] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f reported cstate change: term changed from 0 to 1, leader changed from <none> to e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66). New cstate: current_term: 1 leader_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } health_report { overall_health: UNKNOWN } } }
W20260805 23:15:50.903841 31633 tablet.cc:2369] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:50.905876 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.902293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.23.88.66:46667)
0805 23:15:50.902437 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.902441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.902455 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:50.902537 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.902539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.902598 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.902710 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:50.902924 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.903536 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.904491 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.904730 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.904827 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.905090 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:50.905541 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:50.905644 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.905701 (+ 57us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.905761 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:15:50.911985 31295 raft_consensus.cc:1261] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Refusing update from remote peer e92980f3ebb6487d8f56a5b9bd2f6e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:50.911988 31583 raft_consensus.cc:1261] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Refusing update from remote peer e92980f3ebb6487d8f56a5b9bd2f6e0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:50.912817 31670 consensus_queue.cc:1035] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:50.913122 31674 consensus_queue.cc:1035] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:50.922538 31677 mvcc.cc:205] Tried to move back new op lower bound from 7315340291727446016 to 7315340291594539008. Current Snapshot: MvccSnapshot[applied={T|T < 7315340291727446016}]
I20260805 23:15:50.935379 30332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:50.935602 30332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39178:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:50.941298 30332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cda1e5ea711f4b7a879662344e5df31d: Sending DeleteTablet for 3 replicas of tablet 76431dc616574b0ebb3989153ed60ac3
I20260805 23:15:50.942217 31418 tablet_service.cc:1514] Processing DeleteTablet for tablet 76431dc616574b0ebb3989153ed60ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:50 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:38604
I20260805 23:15:50.942210 31274 tablet_service.cc:1514] Processing DeleteTablet for tablet 76431dc616574b0ebb3989153ed60ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:50 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:33986
I20260805 23:15:50.942787 31687 tablet_replica.cc:330] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: stopping tablet replica
I20260805 23:15:50.942817 31562 tablet_service.cc:1514] Processing DeleteTablet for tablet 76431dc616574b0ebb3989153ed60ac3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:50 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:54364
I20260805 23:15:50.943171 31687 raft_consensus.cc:2229] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:50.943426 31689 tablet_replica.cc:330] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: stopping tablet replica
I20260805 23:15:50.943652 31687 raft_consensus.cc:2258] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:50.943859 31689 raft_consensus.cc:2229] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:50.944495 31689 raft_consensus.cc:2258] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:50.944905 31688 tablet_replica.cc:330] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: stopping tablet replica
I20260805 23:15:50.945485 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30300
I20260805 23:15:50.947822 31687 ts_tablet_manager.cc:1905] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:50.947822 31689 ts_tablet_manager.cc:1905] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:50.950738 31687 ts_tablet_manager.cc:1918] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:50.950738 31689 ts_tablet_manager.cc:1918] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:50.950868 31687 log.cc:1196] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/76431dc616574b0ebb3989153ed60ac3
I20260805 23:15:50.951022 31689 log.cc:1196] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/76431dc616574b0ebb3989153ed60ac3
I20260805 23:15:50.951272 31687 ts_tablet_manager.cc:1939] T 76431dc616574b0ebb3989153ed60ac3 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting consensus metadata
I20260805 23:15:50.951532 31689 ts_tablet_manager.cc:1939] T 76431dc616574b0ebb3989153ed60ac3 P 140d6d27f0214b86a6211f44f74aa97e: Deleting consensus metadata
W20260805 23:15:50.955700 31513 connection.cc:535] server connection from 127.0.0.1:54364 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260805 23:15:50.956084 31370 connection.cc:535] server connection from 127.0.0.1:38604 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:15:50.957345 31688 raft_consensus.cc:2229] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:50.957739 31688 raft_consensus.cc:2258] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:50.960906 31688 ts_tablet_manager.cc:1905] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:50.963171 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.945024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.23.88.125:40405)
0805 23:15:50.945270 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.945276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.953022 (+ 7746us) server_negotiation.cc:407] Connection header received
0805 23:15:50.953164 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.953168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.953231 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.953326 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.954089 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.954637 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.955558 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.955888 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.956030 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.958541 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.958546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.958549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.958581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.960463 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.961015 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.961019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.961021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.961113 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.961426 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.961430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.961432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.961687 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.961810 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.962719 (+ 909us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.962868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260805 23:15:50.965045 31688 ts_tablet_manager.cc:1918] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:50.965184 31688 log.cc:1196] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/76431dc616574b0ebb3989153ed60ac3
I20260805 23:15:50.965580 31688 ts_tablet_manager.cc:1939] T 76431dc616574b0ebb3989153ed60ac3 P ccbd5600c41f4c7296ad486a288199c4: Deleting consensus metadata
W20260805 23:15:50.966969 31226 connection.cc:441] server connection from 127.0.0.1:33986 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33986 (request call id 1) could send its response
I20260805 23:15:50.972149 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.944701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.23.88.124:42685)
0805 23:15:50.944824 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.944829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.962440 (+ 17611us) server_negotiation.cc:407] Connection header received
0805 23:15:50.962554 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.962557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.962619 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.962702 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.963624 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.964353 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.965544 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.965886 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.965992 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.967850 (+ 1858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.967854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.967857 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.967887 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.969864 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.970392 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.970396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.970398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.970491 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.970772 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.970775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.970777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.971006 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.971235 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.971834 (+ 599us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.971957 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:15:50.974344 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.959631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.23.88.125:40405)
0805 23:15:50.959760 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.959765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.961483 (+ 1718us) server_negotiation.cc:407] Connection header received
0805 23:15:50.961697 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.961700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.961748 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.961797 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.964134 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.964886 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.966843 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.967284 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.967352 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.969245 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.969250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.969252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.969281 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.971056 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.971593 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.971597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.971599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.971688 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.972674 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.972678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.972681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.972938 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.973054 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.974043 (+ 989us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.974175 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:50.976820 31495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:50.960730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45676 (local address 127.23.88.124:42685)
0805 23:15:50.960850 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:50.960854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:50.966763 (+ 5909us) server_negotiation.cc:407] Connection header received
0805 23:15:50.967597 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:50.967603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:50.967660 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:50.967737 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:50.968619 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.969447 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.970792 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:50.971294 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:50.971394 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:50.973064 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:50.973070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:50.973076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:50.973108 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:50.974908 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.975473 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.975478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.975479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.975574 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:50.975872 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:50.975875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:50.975877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:50.976155 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:50.976312 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:50.976518 (+ 206us) server_negotiation.cc:299] Negotiation successful
0805 23:15:50.976666 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:15:51.925714 31698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:51.925143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:37399)
0805 23:15:51.925446 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:51.925556 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":87,"threads_started":1}
W20260805 23:15:51.926162 31489 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:46333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:51.929245 31699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:51.928685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:47921)
0805 23:15:51.928992 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:51.929098 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":82,"threads_started":1}
W20260805 23:15:51.929652 31631 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:46333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:51.929979 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:51.929542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:40439)
0805 23:15:51.929791 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:51.929865 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
W20260805 23:15:51.930260 31343 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:46333 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.441303 31701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cda1e5ea711f4b7a879662344e5df31d)
I20260805 23:15:52.441591 31702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cda1e5ea711f4b7a879662344e5df31d)
I20260805 23:15:52.441567 31701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } }
I20260805 23:15:52.441774 31702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } }
I20260805 23:15:52.442664 31701 leader_election.cc:288] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333), 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125:40405)
I20260805 23:15:52.443300 31704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.442836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39684)
0805 23:15:52.443066 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.443179 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
W20260805 23:15:52.443593 30949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.443938 31707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.442832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39672)
0805 23:15:52.443709 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.443789 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":290,"thread_start_us":60,"threads_started":1}
W20260805 23:15:52.444103 31139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.443436 31702 leader_election.cc:288] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333), 997df6ed0f6d456692e7c05692f5e658 (127.23.88.124:42685)
I20260805 23:15:52.444656 31708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.444427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39700)
0805 23:15:52.444526 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.444585 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260805 23:15:52.444765 30949 leader_election.cc:334] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333): Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.445226 31707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.445008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39704)
0805 23:15:52.445110 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.445165 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260805 23:15:52.445349 31139 leader_election.cc:334] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333): Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.447389 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.442832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:40405 (local address 127.0.0.1:60090)
0805 23:15:52.443176 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.443191 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.443313 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.443668 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.443674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.443699 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.443833 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.443845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.444957 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.444960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.446069 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.446079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.446890 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.446893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.446993 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.447023 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.447172 (+ 149us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.447261 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":62,"threads_started":1}
I20260805 23:15:52.447445 31706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.442936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.23.88.125:40405)
0805 23:15:52.443401 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.443406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.443423 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:15:52.443474 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.443476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.443527 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.443984 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.444092 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.444831 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.446193 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.446772 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.446875 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.446942 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.447210 (+ 268us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.447298 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":94,"threads_started":1}
I20260805 23:15:52.447947 30975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "997df6ed0f6d456692e7c05692f5e658" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" is_pre_election: true
I20260805 23:15:52.448160 30975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 997df6ed0f6d456692e7c05692f5e658 in term 1.
I20260805 23:15:52.448568 31704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.443278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:42685 (local address 127.0.0.1:45692)
0805 23:15:52.443581 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.443594 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.443735 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.444523 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.444526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.444543 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.444620 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.444625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.445743 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.445745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.447298 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.447307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.448160 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.448164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.448262 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.448285 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.448389 (+ 104us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.448478 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260805 23:15:52.448559 31141 leader_election.cc:302] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 475ff05cb07a4d92a7b4c17b4dc0eedb, 997df6ed0f6d456692e7c05692f5e658; no voters: cda1e5ea711f4b7a879662344e5df31d
I20260805 23:15:52.448814 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.444009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.23.88.124:42685)
0805 23:15:52.444290 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.444295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.444308 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:15:52.444347 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.444350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.444387 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.444710 (+ 323us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.444789 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.445634 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.447432 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.448014 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.448210 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.448296 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.448524 (+ 228us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.448620 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":220,"thread_start_us":61,"threads_started":1}
I20260805 23:15:52.448911 31701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 23:15:52.448982 31701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cda1e5ea711f4b7a879662344e5df31d)
I20260805 23:15:52.449020 31701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:15:52.449226 31168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "997df6ed0f6d456692e7c05692f5e658" is_pre_election: true
I20260805 23:15:52.450255 31701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } }
I20260805 23:15:52.450495 31168 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 475ff05cb07a4d92a7b4c17b4dc0eedb in current term 2: Already voted for candidate 997df6ed0f6d456692e7c05692f5e658 in this term.
I20260805 23:15:52.450695 31701 leader_election.cc:288] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 election: Requested vote from peers cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333), 475ff05cb07a4d92a7b4c17b4dc0eedb (127.23.88.125:40405)
I20260805 23:15:52.451048 30975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "997df6ed0f6d456692e7c05692f5e658" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb"
I20260805 23:15:52.451195 30975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:15:52.451284 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.450808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39720)
0805 23:15:52.451160 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.451220 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239}
I20260805 23:15:52.451264 30949 leader_election.cc:302] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 475ff05cb07a4d92a7b4c17b4dc0eedb; no voters: 997df6ed0f6d456692e7c05692f5e658, cda1e5ea711f4b7a879662344e5df31d
I20260805 23:15:52.452056 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.451835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39732)
0805 23:15:52.451948 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.451999 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260805 23:15:52.452203 31139 leader_election.cc:334] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333): Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
I20260805 23:15:52.452469 30975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 997df6ed0f6d456692e7c05692f5e658 in term 2.
I20260805 23:15:52.452579 31702 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260805 23:15:52.452816 31141 leader_election.cc:302] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 475ff05cb07a4d92a7b4c17b4dc0eedb, 997df6ed0f6d456692e7c05692f5e658; no voters: cda1e5ea711f4b7a879662344e5df31d
I20260805 23:15:52.453061 31701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:15:52.453286 31701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [term 2 LEADER]: Becoming Leader. State: Replica: 997df6ed0f6d456692e7c05692f5e658, State: Running, Role: LEADER
I20260805 23:15:52.453549 31701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } }
I20260805 23:15:52.454694 31712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997df6ed0f6d456692e7c05692f5e658. Latest consensus state: current_term: 2 leader_uuid: "997df6ed0f6d456692e7c05692f5e658" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:52.454790 31712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:52.455127 31714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:52.457721 31714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:52.458141 31714 catalog_manager.cc:1259] Loaded cluster ID: 84c7ee6b29114756b0299c5d6d2c4439
I20260805 23:15:52.458217 31714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:52.458840 31714 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:52.459316 31714 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Loaded TSK: 0
I20260805 23:15:52.460019 31714 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:15:52.508163 31158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:52.511660 31158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:52.515209 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.514893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39762)
0805 23:15:52.515030 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.515124 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260805 23:15:52.515388 30975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [term 2 FOLLOWER]: Refusing update from remote peer 997df6ed0f6d456692e7c05692f5e658: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260805 23:15:52.516057 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.515816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.0.0.1:39774)
0805 23:15:52.515915 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.515976 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260805 23:15:52.516021 31712 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260805 23:15:52.516204 31139 consensus_peers.cc:487] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 -> Peer cda1e5ea711f4b7a879662344e5df31d (127.23.88.126:46333): Couldn't send request to peer cda1e5ea711f4b7a879662344e5df31d. Status: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260805 23:15:52.517804 31702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 997df6ed0f6d456692e7c05692f5e658. Latest consensus state: current_term: 2 leader_uuid: "997df6ed0f6d456692e7c05692f5e658" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:52.517979 31702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:52.519297 31712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "997df6ed0f6d456692e7c05692f5e658" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:52.519461 31712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:52.519824 31702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "997df6ed0f6d456692e7c05692f5e658" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "cda1e5ea711f4b7a879662344e5df31d" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 46333 } } peers { permanent_uuid: "475ff05cb07a4d92a7b4c17b4dc0eedb" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 40405 } attrs { promote: false } } peers { permanent_uuid: "997df6ed0f6d456692e7c05692f5e658" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 42685 } attrs { promote: false } } }
I20260805 23:15:52.519981 31702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475ff05cb07a4d92a7b4c17b4dc0eedb [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:15:52.530148 31724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.526152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46667 (local address 127.0.0.1:38616)
0805 23:15:52.526425 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.526439 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.526518 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.526777 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.526780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.526796 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.526897 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.526902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.527835 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.527839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.528941 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.528951 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.529675 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.529770 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.529797 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.529896 (+ 99us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.529983 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":131,"threads_started":1}
I20260805 23:15:52.530383 31705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.525401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35477 (local address 127.0.0.1:54366)
0805 23:15:52.525516 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.525532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.525617 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.527011 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.527014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.527030 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.527156 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.527162 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.528183 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.528186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.529051 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.529062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.529957 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529963 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.530068 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.530093 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.530144 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.530237 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260805 23:15:52.530578 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.526298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.23.88.66:46667)
0805 23:15:52.526533 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.526536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.526551 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:52.526621 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.526627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.526684 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.526772 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.527029 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.527737 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529078 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.529548 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529673 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.529744 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.530327 (+ 583us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.530416 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260805 23:15:52.530891 31707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.525579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45877 (local address 127.0.0.1:34002)
0805 23:15:52.525832 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.525847 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.525938 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.527494 (+ 1556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.527498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.527517 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.527605 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.527611 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.528492 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.528495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.529633 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.529643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.530577 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.530580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.530675 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.530699 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.530750 (+ 51us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.530813 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260805 23:15:52.531404 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.525815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.23.88.67:35477)
0805 23:15:52.526766 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.526773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.526788 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:52.526850 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.526853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.526902 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.527003 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.527291 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.528063 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529179 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.529814 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.531026 (+ 1212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.531132 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.531176 (+ 44us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.531267 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":869,"thread_start_us":107,"threads_started":1}
I20260805 23:15:52.531533 31418 tablet_service.cc:1467] Processing CreateTablet for tablet 13498c7083fa49daa125febd187b3c52 (DEFAULT_TABLE table=test-table [id=377466d7b8734c3bb58a8a7aaf68e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:52.531611 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.525850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34002 (local address 127.23.88.65:45877)
0805 23:15:52.527257 (+ 1407us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.527261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.527275 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:52.527322 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.527326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.527377 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.527462 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.527742 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.528373 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.529807 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.530428 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.531285 (+ 857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.531359 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.531394 (+ 35us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.531480 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1314,"thread_start_us":89,"threads_started":1}
I20260805 23:15:52.531859 31418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13498c7083fa49daa125febd187b3c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:52.531891 31562 tablet_service.cc:1467] Processing CreateTablet for tablet 13498c7083fa49daa125febd187b3c52 (DEFAULT_TABLE table=test-table [id=377466d7b8734c3bb58a8a7aaf68e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:52.532035 31274 tablet_service.cc:1467] Processing CreateTablet for tablet 13498c7083fa49daa125febd187b3c52 (DEFAULT_TABLE table=test-table [id=377466d7b8734c3bb58a8a7aaf68e5bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:52.532169 31562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13498c7083fa49daa125febd187b3c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:52.532287 31274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13498c7083fa49daa125febd187b3c52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:52.534199 31726 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Bootstrap starting.
I20260805 23:15:52.534700 31728 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4: Bootstrap starting.
I20260805 23:15:52.535316 31726 tablet_bootstrap.cc:654] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:52.535293 31727 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Bootstrap starting.
I20260805 23:15:52.535702 31728 tablet_bootstrap.cc:654] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:52.536222 31727 tablet_bootstrap.cc:654] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:52.536736 31726 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: No bootstrap required, opened a new log
I20260805 23:15:52.536851 31726 ts_tablet_manager.cc:1397] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:52.537236 31726 raft_consensus.cc:348] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.537359 31727 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: No bootstrap required, opened a new log
I20260805 23:15:52.537360 31726 raft_consensus.cc:374] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:52.537400 31726 raft_consensus.cc:729] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92980f3ebb6487d8f56a5b9bd2f6e0f, State: Initialized, Role: FOLLOWER
I20260805 23:15:52.537427 31727 ts_tablet_manager.cc:1397] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:52.537546 31726 consensus_queue.cc:260] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.537916 31726 ts_tablet_manager.cc:1428] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:52.538036 31727 raft_consensus.cc:348] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.538209 31727 raft_consensus.cc:374] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:52.538255 31727 raft_consensus.cc:729] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 140d6d27f0214b86a6211f44f74aa97e, State: Initialized, Role: FOLLOWER
I20260805 23:15:52.538429 31727 consensus_queue.cc:260] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.538775 31727 ts_tablet_manager.cc:1428] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:15:52.540369 31728 tablet_bootstrap.cc:492] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4: No bootstrap required, opened a new log
I20260805 23:15:52.540724 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.538743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:51745)
0805 23:15:52.540550 (+ 1807us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.540618 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1736,"thread_start_us":118,"threads_started":1}
I20260805 23:15:52.541205 31728 ts_tablet_manager.cc:1397] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260805 23:15:52.541610 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.541343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:39799)
0805 23:15:52.541472 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.541534 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260805 23:15:52.541929 31488 heartbeater.cc:503] Master 127.23.88.124:42685 was elected leader, sending a full tablet report...
I20260805 23:15:52.542308 31728 raft_consensus.cc:348] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.542428 31728 raft_consensus.cc:374] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:52.542474 31728 raft_consensus.cc:729] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbd5600c41f4c7296ad486a288199c4, State: Initialized, Role: FOLLOWER
I20260805 23:15:52.542604 31728 consensus_queue.cc:260] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.542806 31630 heartbeater.cc:503] Master 127.23.88.124:42685 was elected leader, sending a full tablet report...
I20260805 23:15:52.542975 31728 ts_tablet_manager.cc:1428] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260805 23:15:52.544145 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.539053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:33477)
0805 23:15:52.543948 (+ 4895us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.544012 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4828,"thread_start_us":122,"threads_started":1}
I20260805 23:15:52.544801 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.544582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:44547)
0805 23:15:52.544693 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.544742 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260805 23:15:52.544832 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.544392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:51581)
0805 23:15:52.544666 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.544729 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":138,"threads_started":1}
I20260805 23:15:52.545353 31342 heartbeater.cc:503] Master 127.23.88.124:42685 was elected leader, sending a full tablet report...
I20260805 23:15:52.545869 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.545633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:58951)
0805 23:15:52.545754 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.545807 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260805 23:15:52.554167 31733 raft_consensus.cc:482] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:52.554340 31733 raft_consensus.cc:504] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.555744 31733 leader_election.cc:288] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877), e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:52.561352 31725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.556394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:41759 (local address 127.23.88.66:46667)
0805 23:15:52.556541 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.556565 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.556737 (+ 172us) server_negotiation.cc:407] Connection header received
0805 23:15:52.556917 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.556923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.556977 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.557036 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.557323 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.558092 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.559961 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.560586 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.560713 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.560765 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.561138 (+ 373us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.561216 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:52.561352 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.556163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:57793 (local address 127.23.88.65:45877)
0805 23:15:52.556349 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.556353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.556368 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:52.556653 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.556656 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.556729 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.556807 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:52.557186 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.557966 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.559998 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.560586 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.560713 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.560765 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.561138 (+ 373us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.561216 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:15:52.561409 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.555959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45877 (local address 127.23.88.67:57793)
0805 23:15:52.556125 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.556141 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.556480 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.556826 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.556831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.556855 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.557051 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.557061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.558094 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.558097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.559827 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.559839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.560754 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.560758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.560949 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.561014 (+ 65us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.561107 (+ 93us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.561256 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":273}
I20260805 23:15:52.561424 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.556222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:46667 (local address 127.23.88.67:41759)
0805 23:15:52.556597 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.556614 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:52.556787 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:52.557073 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:52.557076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:52.557097 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:52.557216 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.557223 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.558177 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.558180 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.559827 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:52.559839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.560754 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.560758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:52.560945 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.561018 (+ 73us) client_negotiation.cc:769] Sending connection context
0805 23:15:52.561107 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 23:15:52.561256 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":249,"thread_start_us":165,"threads_started":1}
I20260805 23:15:52.561851 31295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13498c7083fa49daa125febd187b3c52" candidate_uuid: "140d6d27f0214b86a6211f44f74aa97e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbd5600c41f4c7296ad486a288199c4" is_pre_election: true
I20260805 23:15:52.561996 31295 raft_consensus.cc:2454] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 140d6d27f0214b86a6211f44f74aa97e in term 0.
I20260805 23:15:52.562403 31516 leader_election.cc:302] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140d6d27f0214b86a6211f44f74aa97e, ccbd5600c41f4c7296ad486a288199c4; no voters:
I20260805 23:15:52.562479 31439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13498c7083fa49daa125febd187b3c52" candidate_uuid: "140d6d27f0214b86a6211f44f74aa97e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" is_pre_election: true
I20260805 23:15:52.562696 31733 raft_consensus.cc:2781] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:52.562727 31439 raft_consensus.cc:2454] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 140d6d27f0214b86a6211f44f74aa97e in term 0.
I20260805 23:15:52.562772 31733 raft_consensus.cc:482] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:52.562829 31733 raft_consensus.cc:3037] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:52.563959 31733 raft_consensus.cc:504] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.564327 31733 leader_election.cc:288] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [CANDIDATE]: Term 1 election: Requested vote from peers ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877), e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667)
I20260805 23:15:52.564652 31295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13498c7083fa49daa125febd187b3c52" candidate_uuid: "140d6d27f0214b86a6211f44f74aa97e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccbd5600c41f4c7296ad486a288199c4"
I20260805 23:15:52.564803 31295 raft_consensus.cc:3037] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:52.564770 31439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13498c7083fa49daa125febd187b3c52" candidate_uuid: "140d6d27f0214b86a6211f44f74aa97e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f"
I20260805 23:15:52.564885 31439 raft_consensus.cc:3037] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:52.566009 31295 raft_consensus.cc:2454] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 140d6d27f0214b86a6211f44f74aa97e in term 1.
I20260805 23:15:52.566074 31439 raft_consensus.cc:2454] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 140d6d27f0214b86a6211f44f74aa97e in term 1.
I20260805 23:15:52.566372 31516 leader_election.cc:302] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 140d6d27f0214b86a6211f44f74aa97e, ccbd5600c41f4c7296ad486a288199c4; no voters:
I20260805 23:15:52.566574 31733 raft_consensus.cc:2781] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:52.566824 31733 raft_consensus.cc:686] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 1 LEADER]: Becoming Leader. State: Replica: 140d6d27f0214b86a6211f44f74aa97e, State: Running, Role: LEADER
I20260805 23:15:52.567118 31733 consensus_queue.cc:237] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } }
I20260805 23:15:52.567855 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.567573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:52033)
0805 23:15:52.567726 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.567789 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260805 23:15:52.569768 31157 catalog_manager.cc:5495] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e reported cstate change: term changed from 0 to 1, leader changed from <none> to 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67). New cstate: current_term: 1 leader_uuid: "140d6d27f0214b86a6211f44f74aa97e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "140d6d27f0214b86a6211f44f74aa97e" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:52.588188 31721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.584530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54368 (local address 127.23.88.67:35477)
0805 23:15:52.584647 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:52.584650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:52.584664 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:52.584769 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:52.584771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:52.584828 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:52.584916 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:52.585125 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.585654 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.586491 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:52.586767 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:52.586947 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:52.587235 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:52.587740 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:52.587828 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:52.587896 (+ 68us) server_negotiation.cc:299] Negotiation successful
0805 23:15:52.587982 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260805 23:15:52.591253 31295 raft_consensus.cc:1261] T 13498c7083fa49daa125febd187b3c52 P ccbd5600c41f4c7296ad486a288199c4 [term 1 FOLLOWER]: Refusing update from remote peer 140d6d27f0214b86a6211f44f74aa97e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:52.591477 31439 raft_consensus.cc:1261] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Refusing update from remote peer 140d6d27f0214b86a6211f44f74aa97e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:52.591904 31733 consensus_queue.cc:1035] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccbd5600c41f4c7296ad486a288199c4" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:52.592113 31739 consensus_queue.cc:1035] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e92980f3ebb6487d8f56a5b9bd2f6e0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:52.594388 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.594059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:47399)
0805 23:15:52.594225 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.594285 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260805 23:15:52.594601 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.594319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:45519)
0805 23:15:52.594476 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.594532 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260805 23:15:52.598838 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.598567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.65:34315)
0805 23:15:52.598704 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.598759 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260805 23:15:52.599447 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.598539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.67:60451)
0805 23:15:52.599289 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.599362 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":679}
I20260805 23:15:52.604640 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:52.604333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:46333 (local address 127.23.88.66:44817)
0805 23:15:52.604486 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.604551 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:46333: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260805 23:15:52.606014 31158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:52.606189 31158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:52.610769 31158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 997df6ed0f6d456692e7c05692f5e658: Sending DeleteTablet for 3 replicas of tablet 13498c7083fa49daa125febd187b3c52
I20260805 23:15:52.611526 31562 tablet_service.cc:1514] Processing DeleteTablet for tablet 13498c7083fa49daa125febd187b3c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:52 UTC) from {username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:54366
I20260805 23:15:52.611742 31418 tablet_service.cc:1514] Processing DeleteTablet for tablet 13498c7083fa49daa125febd187b3c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:52 UTC) from {username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:38616
I20260805 23:15:52.612345 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 31115
I20260805 23:15:52.612337 31274 tablet_service.cc:1514] Processing DeleteTablet for tablet 13498c7083fa49daa125febd187b3c52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:52 UTC) from {username='oryx', principal='oryx/127.23.88.124@KRBTEST.COM'} at 127.0.0.1:34002
I20260805 23:15:52.614292 31753 tablet_replica.cc:330] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: stopping tablet replica
I20260805 23:15:52.614436 31753 raft_consensus.cc:2229] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:52.614579 31753 raft_consensus.cc:2258] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:52.614835 31752 tablet_replica.cc:330] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: stopping tablet replica
I20260805 23:15:52.614944 31752 raft_consensus.cc:2229] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:52.615298 31752 raft_consensus.cc:2258] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:52.615916 31753 ts_tablet_manager.cc:1905] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:52.618465 31753 ts_tablet_manager.cc:1918] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:52.618564 31753 log.cc:1196] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/13498c7083fa49daa125febd187b3c52
I20260805 23:15:52.618898 31753 ts_tablet_manager.cc:1939] T 13498c7083fa49daa125febd187b3c52 P e92980f3ebb6487d8f56a5b9bd2f6e0f: Deleting consensus metadata
I20260805 23:15:52.619427 31752 ts_tablet_manager.cc:1905] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:52.619827 31142 catalog_manager.cc:4841] TS e92980f3ebb6487d8f56a5b9bd2f6e0f (127.23.88.66:46667): tablet 13498c7083fa49daa125febd187b3c52 (table test-table [id=377466d7b8734c3bb58a8a7aaf68e5bb]) successfully deleted
I20260805 23:15:52.622071 31752 ts_tablet_manager.cc:1918] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:52.622156 31752 log.cc:1196] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/13498c7083fa49daa125febd187b3c52
I20260805 23:15:52.622433 31752 ts_tablet_manager.cc:1939] T 13498c7083fa49daa125febd187b3c52 P 140d6d27f0214b86a6211f44f74aa97e: Deleting consensus metadata
I20260805 23:15:52.623255 31140 catalog_manager.cc:4841] TS 140d6d27f0214b86a6211f44f74aa97e (127.23.88.67:35477): tablet 13498c7083fa49daa125febd187b3c52 (table test-table [id=377466d7b8734c3bb58a8a7aaf68e5bb]) successfully deleted
W20260805 23:15:52.624140 31707 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:52.623775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45877 (local address 127.0.0.1:34014)
0805 23:15:52.623975 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:52.624035 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45877: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":104}
I20260805 23:15:52.624413 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 31351
W20260805 23:15:52.624401 31142 catalog_manager.cc:4584] TS ccbd5600c41f4c7296ad486a288199c4 (127.23.88.65:45877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 13498c7083fa49daa125febd187b3c52: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45877: connect: Connection reset by peer (error 104)
I20260805 23:15:52.633060 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 31496
I20260805 23:15:52.640657 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30822
I20260805 23:15:52.648001 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 30928
2026-08-05T23:15:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5586 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:52 dist-test-slave-vkjp krb5kdc[31756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:52 dist-test-slave-vkjp krb5kdc[31756](info): set up 2 sockets
Aug 05 23:15:52 dist-test-slave-vkjp krb5kdc[31756](info): commencing operation
krb5kdc: starting...
W20260805 23:15:53.725893 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:53 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab.
Entry for principal oryx/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab.
I20260805 23:15:53.854086 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37819
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab
--principal=oryx/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:53.968897 31769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:53.969213 31769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:53.969259 31769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:53.973459 31769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:53.973546 31769 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260805 23:15:53.973563 31769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:53.973585 31769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:53.973603 31769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:53.973618 31769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:53.979547 31769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37819
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.126.keytab
--principal=oryx/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:53.980840 31769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:53.981997 31769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:53.991835 31775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:53.991839 31776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:53.991835 31778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:53.992350 31769 server_base.cc:1034] running on GCE node
I20260805 23:15:53.993072 31769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:53.994257 31769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:53.995435 31769 hybrid_clock.cc:648] HybridClock initialized: now 1785971753995414 us; error 34 us; skew 500 ppm
Aug 05 23:15:53 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971753, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:54.002736 31769 init.cc:377] Logged in from keytab as oryx/127.23.88.126@KRBTEST.COM (short username oryx)
I20260805 23:15:54.003849 31769 webserver.cc:460] Webserver started at http://127.23.88.126:44815/ using document root <none> and password file <none>
I20260805 23:15:54.004464 31769 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:54.004530 31769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:54.004761 31769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:54.006996 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "0b289cd10540404cb73483198827c8f2"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d3ee2ceb6560f057855c9e8be38e9fa6"
server_key_iv: "990d183dd03356a33a6f5ba75910f2f2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.007706 31769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "0b289cd10540404cb73483198827c8f2"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d3ee2ceb6560f057855c9e8be38e9fa6"
server_key_iv: "990d183dd03356a33a6f5ba75910f2f2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.011806 31769 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:54.014540 31786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.015998 31769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:54.016146 31769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "0b289cd10540404cb73483198827c8f2"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d3ee2ceb6560f057855c9e8be38e9fa6"
server_key_iv: "990d183dd03356a33a6f5ba75910f2f2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.016284 31769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:54.031771 31769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:54.035058 31769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:54.035359 31769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:54.044185 31769 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:37819
I20260805 23:15:54.044211 31850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:37819 every 8 connection(s)
I20260805 23:15:54.045178 31769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:54.048132 31852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:54.051433 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 31769
I20260805 23:15:54.051563 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:54.051865 23905 external_mini_cluster.cc:1428] Setting key f9c406c14f4ada7daf76b4a1c9a4b58c
I20260805 23:15:54.054731 31852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Bootstrap starting.
I20260805 23:15:54.057550 31852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:54.058493 31852 log.cc:824] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:54.060783 31852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: No bootstrap required, opened a new log
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:54.064500 31852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } }
I20260805 23:15:54.064738 31852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:54.064800 31852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b289cd10540404cb73483198827c8f2, State: Initialized, Role: FOLLOWER
I20260805 23:15:54.065248 31852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } }
I20260805 23:15:54.065375 31852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:54.065466 31852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:54.065596 31852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:54.066646 31852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } }
I20260805 23:15:54.067003 31852 leader_election.cc:302] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b289cd10540404cb73483198827c8f2; no voters:
I20260805 23:15:54.067430 31852 leader_election.cc:288] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:54.067508 31857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:54.067649 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.053359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.23.88.126:37819)
0805 23:15:54.053835 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.053851 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.053889 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:15:54.054530 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.054546 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.054942 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.055299 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.056256 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.057296 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.058507 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.059047 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.059252 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.062228 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.062233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.062248 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.062278 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.064170 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.064659 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.064664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.064669 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.064861 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.065173 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.065176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.065178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.065749 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.065965 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.066310 (+ 345us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.066637 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":166,"threads_started":1}
I20260805 23:15:54.067699 31857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [term 1 LEADER]: Becoming Leader. State: Replica: 0b289cd10540404cb73483198827c8f2, State: Running, Role: LEADER
I20260805 23:15:54.068786 31852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:54.069062 31857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } }
I20260805 23:15:54.070825 31858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b289cd10540404cb73483198827c8f2. Latest consensus state: current_term: 1 leader_uuid: "0b289cd10540404cb73483198827c8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } } }
I20260805 23:15:54.070998 31858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:54.071116 31859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b289cd10540404cb73483198827c8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b289cd10540404cb73483198827c8f2" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37819 } } }
I20260805 23:15:54.071626 31859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2 [sys.catalog]: This master's current role is: LEADER
W20260805 23:15:54.073817 31872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:54.073886 31872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:54.073971 31873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:54.074713 31873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:54.080859 31873 catalog_manager.cc:1347] Generated new cluster ID: fa2773108b2e47d9a4c9c354a6b22297
I20260805 23:15:54.080950 31873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:54.087802 31873 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:54.089221 31873 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:54.096206 31873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Generated new TSK 0
I20260805 23:15:54.097136 31873 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
I20260805 23:15:54.135047 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37819
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:54.252475 31879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:54.252770 31879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:54.252816 31879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:54.257004 31879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:54.257078 31879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:54.257119 31879 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:54.257197 31879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:54.262768 31879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:54.263931 31879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:54.264972 31879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:15:54.275098 31879 server_base.cc:1034] running on GCE node
W20260805 23:15:54.274995 31884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:54.275012 31885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:54.274995 31887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:54.275822 31879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:54.276466 31879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:54.277688 31879 hybrid_clock.cc:648] HybridClock initialized: now 1785971754277674 us; error 24 us; skew 500 ppm
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:54.280571 31879 init.cc:377] Logged in from keytab as oryx/127.23.88.65@KRBTEST.COM (short username oryx)
I20260805 23:15:54.281474 31879 webserver.cc:460] Webserver started at http://127.23.88.65:38397/ using document root <none> and password file <none>
I20260805 23:15:54.282058 31879 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:54.282120 31879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:54.282343 31879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:54.284550 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "cc45e232208949ccb94739a0f9974e6d"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "7286ee125f0e6652c947baf6dd0aa9e6"
server_key_iv: "fab8b0e0e88dcb0d61813bab4933c0a2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.285151 31879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "cc45e232208949ccb94739a0f9974e6d"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "7286ee125f0e6652c947baf6dd0aa9e6"
server_key_iv: "fab8b0e0e88dcb0d61813bab4933c0a2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.289168 31879 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:54.291920 31894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.293255 31879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:54.293386 31879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "cc45e232208949ccb94739a0f9974e6d"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "7286ee125f0e6652c947baf6dd0aa9e6"
server_key_iv: "fab8b0e0e88dcb0d61813bab4933c0a2"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.293516 31879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:54.313040 31879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:54.316332 31879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:54.316571 31879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:54.318126 31879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:54.318208 31879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.318286 31879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:54.318329 31879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:54.329663 31879 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:37881
I20260805 23:15:54.329728 32014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:37881 every 8 connection(s)
I20260805 23:15:54.330893 31879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:54.333504 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.319177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:57175 (local address 127.23.88.126:37819)
0805 23:15:54.319315 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.319319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.320612 (+ 1293us) server_negotiation.cc:407] Connection header received
0805 23:15:54.321876 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.321879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.321964 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.322056 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.323812 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.324359 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.325728 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.325993 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.326114 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.329475 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.329480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.329483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.329517 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.331310 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.331953 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.331958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.331963 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.332062 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.332386 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.332388 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.332389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.332594 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.332671 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.333207 (+ 536us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.333330 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:15:54.334105 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 31879
I20260805 23:15:54.334232 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:54.334429 31903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.319792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.65:57175)
0805 23:15:54.320454 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.320500 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.321634 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.322252 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.322262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.322743 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.323513 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.323534 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.324520 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.324526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.325603 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.325613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.326176 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.326179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.326354 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.327639 (+ 1285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.327656 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.329217 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.331455 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.331461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.331475 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.331815 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.332154 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.332156 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.332158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.332292 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.332686 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.332691 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.332927 (+ 236us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.333092 (+ 165us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.333350 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":169,"threads_started":1}
I20260805 23:15:54.334620 23905 external_mini_cluster.cc:1428] Setting key 58acc43875244c78e36d90dcf72083cc
I20260805 23:15:54.338397 32015 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37819
I20260805 23:15:54.338585 32015 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:54.339263 32015 heartbeater.cc:511] Master 127.23.88.126:37819 requested a full tablet report, sending...
I20260805 23:15:54.340931 31802 ts_manager.cc:194] Registered new tserver with Master: cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881)
I20260805 23:15:54.342084 31802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:57175
WARNING: no policy specified for oryx/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.66@KRBTEST.COM" created.
I20260805 23:15:54.349455 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.339557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:43877 (local address 127.23.88.126:37819)
0805 23:15:54.339698 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.339705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.339920 (+ 215us) server_negotiation.cc:407] Connection header received
0805 23:15:54.340078 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.340081 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.340137 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.340224 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.343398 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.343920 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.344747 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.344979 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.345090 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.346354 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.346357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.346359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.346382 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.347738 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.348202 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.348206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.348207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.348280 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.348562 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.348567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.348568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.348751 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.348862 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.349165 (+ 303us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.349283 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:15:54.349470 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.339444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.65:43877)
0805 23:15:54.339810 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.339823 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.339965 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.342683 (+ 2718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.342686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.342989 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.343260 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.343270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.344062 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.344065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.344620 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.344628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.345111 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.345116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.345218 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.345796 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.345808 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.346197 (+ 389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.347864 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.347868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.347870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.348084 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.348375 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.348379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.348381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.348468 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.348854 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.348857 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.348975 (+ 118us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.349104 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.349272 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":35,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
I20260805 23:15:54.367542 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37819
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:54.481456 32027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:54.481786 32027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:54.481850 32027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:54.486080 32027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:54.486156 32027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:54.486197 32027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:54.486231 32027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:54.491683 32027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:54.492789 32027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:54.493937 32027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:54.504202 32033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:54.504202 32032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:54.504279 32027 server_base.cc:1034] running on GCE node
W20260805 23:15:54.504195 32035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:54.504897 32027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:54.505611 32027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:54.506801 32027 hybrid_clock.cc:648] HybridClock initialized: now 1785971754506767 us; error 48 us; skew 500 ppm
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:54.509815 32027 init.cc:377] Logged in from keytab as oryx/127.23.88.66@KRBTEST.COM (short username oryx)
I20260805 23:15:54.510758 32027 webserver.cc:460] Webserver started at http://127.23.88.66:45967/ using document root <none> and password file <none>
I20260805 23:15:54.511438 32027 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:54.511536 32027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:54.511786 32027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:54.513967 32027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "d61ee8e651624026bd77bab9fd78d821"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "4b2a576571bc204c69872b5a40afe19e"
server_key_iv: "39bf5567b41774e4635a4facd28893ac"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.514621 32027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "d61ee8e651624026bd77bab9fd78d821"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "4b2a576571bc204c69872b5a40afe19e"
server_key_iv: "39bf5567b41774e4635a4facd28893ac"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.518533 32027 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260805 23:15:54.521337 32042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.522603 32027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:54.522753 32027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "d61ee8e651624026bd77bab9fd78d821"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "4b2a576571bc204c69872b5a40afe19e"
server_key_iv: "39bf5567b41774e4635a4facd28893ac"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.522874 32027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:54.542292 32027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:54.546015 32027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:54.546239 32027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:54.547904 32027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:54.547983 32027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.548058 32027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:54.548115 32027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:54.559327 32027 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:40531
I20260805 23:15:54.559399 32166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:40531 every 8 connection(s)
I20260805 23:15:54.560510 32027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:54.562960 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.548684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:53109 (local address 127.23.88.126:37819)
0805 23:15:54.548820 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.548824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.549968 (+ 1144us) server_negotiation.cc:407] Connection header received
0805 23:15:54.551051 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.551057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.551151 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.551214 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.552956 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.553881 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.555361 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.555610 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.555731 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.559244 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.559248 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.559250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.559306 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.560837 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.561508 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.561512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.561514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.561614 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.561942 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.561944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.561945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.562132 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.562209 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.562709 (+ 500us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.562820 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:54.563855 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.549270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.66:53109)
0805 23:15:54.549802 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.549848 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.550773 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.551687 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.551694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.552115 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.552763 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.552782 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.554024 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.554027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.555224 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.555234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.555793 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.555797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.555986 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.557226 (+ 1240us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.557244 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.559002 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.560970 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.560976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.561001 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.561365 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.561710 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.561713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.561714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.561847 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.562220 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.562225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.562479 (+ 254us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.562661 (+ 182us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.562921 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":169,"threads_started":1}
I20260805 23:15:54.565099 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 32027
I20260805 23:15:54.565258 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:15:54.565567 23905 external_mini_cluster.cc:1428] Setting key 61007d4f5b960a6643ad01706a85cbb4
I20260805 23:15:54.565979 32167 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37819
I20260805 23:15:54.566138 32167 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:54.566711 32167 heartbeater.cc:511] Master 127.23.88.126:37819 requested a full tablet report, sending...
I20260805 23:15:54.569321 31803 ts_manager.cc:194] Registered new tserver with Master: d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531)
I20260805 23:15:54.570207 31803 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:53109
WARNING: no policy specified for oryx/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:15:54.575855 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.567649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:40187 (local address 127.23.88.126:37819)
0805 23:15:54.567758 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.567761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.568049 (+ 288us) server_negotiation.cc:407] Connection header received
0805 23:15:54.568199 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.568202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.568253 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.568338 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.568792 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.569327 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.570277 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.570595 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.570689 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.572396 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.572400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.572403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.572429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.573990 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.574509 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.574513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.574515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.574599 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.574888 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.574893 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.574894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.575189 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.575301 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.575567 (+ 266us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.575687 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 23:15:54.575874 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.567687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.66:40187)
0805 23:15:54.567965 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.567977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.568095 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.568355 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.568358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.568577 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.568677 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.568683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.569487 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.569491 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.570161 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.570170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.570948 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.570951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.571050 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.571777 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.571792 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.572231 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.574111 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.574115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.574118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.574398 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.574698 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.574701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.574703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.574794 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.575289 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.575292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.575397 (+ 105us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.575508 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.575672 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":100,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
I20260805 23:15:54.596865 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:37819
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:36153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:54.714916 32176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:54.715297 32176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:54.715370 32176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:54.719628 32176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:54.719705 32176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:54.719746 32176 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260805 23:15:54.719816 32176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:54.725874 32176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.88.126:37819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:54.727232 32176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:54.728317 32176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:54.738687 32181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:54.738781 32176 server_base.cc:1034] running on GCE node
W20260805 23:15:54.738720 32182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:54.738710 32184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:54.739360 32176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:54.740082 32176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:54.741250 32176 hybrid_clock.cc:648] HybridClock initialized: now 1785971754741237 us; error 27 us; skew 500 ppm
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:54.744138 32176 init.cc:377] Logged in from keytab as oryx/127.23.88.67@KRBTEST.COM (short username oryx)
I20260805 23:15:54.745045 32176 webserver.cc:460] Webserver started at http://127.23.88.67:32905/ using document root <none> and password file <none>
I20260805 23:15:54.745589 32176 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:54.745649 32176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:54.745821 32176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:54.748018 32176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "7d6de65e1e71410bb07f0b82f068c088"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d08e26538f193be24cc3546c19415e0d"
server_key_iv: "9389159f299e89dc35d8b07e5ca896d6"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.748570 32176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "7d6de65e1e71410bb07f0b82f068c088"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d08e26538f193be24cc3546c19415e0d"
server_key_iv: "9389159f299e89dc35d8b07e5ca896d6"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.752563 32176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:15:54.755208 32191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.756596 32176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:54.756731 32176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "7d6de65e1e71410bb07f0b82f068c088"
format_stamp: "Formatted at 2026-08-05 23:15:54 on dist-test-slave-vkjp"
server_key: "d08e26538f193be24cc3546c19415e0d"
server_key_iv: "9389159f299e89dc35d8b07e5ca896d6"
server_key_version: "encryptionkey@0"
I20260805 23:15:54.756852 32176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:54.770319 32176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:54.773479 32176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:54.773708 32176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:54.775313 32176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:54.775398 32176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:54.775481 32176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:54.775525 32176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:54.786860 32176 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:36601
I20260805 23:15:54.786934 32312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:36601 every 8 connection(s)
I20260805 23:15:54.787830 32176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:54.789649 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.776125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:40273 (local address 127.23.88.126:37819)
0805 23:15:54.776287 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.776293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.777238 (+ 945us) server_negotiation.cc:407] Connection header received
0805 23:15:54.778288 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.778294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.778365 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.778434 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.780032 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.780953 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.782234 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.782640 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.782778 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.785541 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.785545 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.785549 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.785582 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.787368 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.787915 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.787921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.787923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.788031 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.788476 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.788479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.788481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.788726 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.788822 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.789327 (+ 505us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.789471 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:15:54.790450 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.776581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.67:40273)
0805 23:15:54.777055 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.777107 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.778030 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.778770 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.778776 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.779288 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.779856 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.779871 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.781139 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.781143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.782083 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.782099 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.782837 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.782842 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.783120 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.783793 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.783809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.785336 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.787496 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.787502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.787517 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.787787 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.788129 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.788132 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.788134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.788345 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.788827 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.788837 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.789089 (+ 252us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.789281 (+ 192us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.789611 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":137,"threads_started":1}
I20260805 23:15:54.791751 32313 heartbeater.cc:347] Connected to a master server at 127.23.88.126:37819
I20260805 23:15:54.791918 32313 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:54.792438 32313 heartbeater.cc:511] Master 127.23.88.126:37819 requested a full tablet report, sending...
I20260805 23:15:54.793601 31802 ts_manager.cc:194] Registered new tserver with Master: 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:54.794288 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 32176
I20260805 23:15:54.794430 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:54.794416 31802 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:40273
I20260805 23:15:54.794757 23905 external_mini_cluster.cc:1428] Setting key faa40c79a53311c866e97e46336b7427
I20260805 23:15:54.796883 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:15:54.802793 32319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.793990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.23.88.67:41781)
0805 23:15:54.795280 (+ 1290us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.795295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.795421 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.795870 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.795873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.796110 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:54.796292 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.796302 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.797305 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.797308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.797965 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.797975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.798459 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.798463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.798555 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.799130 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:54.799144 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:54.799501 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:54.801212 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.801217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.801219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.801429 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.801720 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:54.801724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:54.801726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:54.801817 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:54.802206 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:54.802209 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:54.802312 (+ 103us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.802425 (+ 113us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.802584 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"thread_start_us":220,"threads_started":1}
I20260805 23:15:54.802786 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.794035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:41781 (local address 127.23.88.126:37819)
0805 23:15:54.794157 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.794162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.795378 (+ 1216us) server_negotiation.cc:407] Connection header received
0805 23:15:54.795516 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.795519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.795569 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.795650 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.796466 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.797166 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.798088 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.798338 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.798450 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.799650 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.799653 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.799656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.799682 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.801097 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.801535 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.801539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.801541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.801628 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.801910 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.801914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.801916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.802109 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.802221 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.802490 (+ 269us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.802611 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:15:54.823076 31855 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:54.816466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55818 (local address 127.23.88.126:37819)
0805 23:15:54.816607 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.816610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.816624 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:54.816670 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.816672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.816714 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.816788 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.817591 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.818119 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.819034 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.819330 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.819414 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.822875 (+ 3461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55818: BlockingRecv error: recv got EOF from 127.0.0.1:55818 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 05 23:15:54 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971754, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:54.833155 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.824604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55834 (local address 127.23.88.126:37819)
0805 23:15:54.824773 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.824776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.824841 (+ 65us) server_negotiation.cc:407] Connection header received
0805 23:15:54.824952 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.824954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.824997 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.825091 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:54.825710 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.826216 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.827218 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.827487 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.827572 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.830015 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:54.830018 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:54.830021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:54.830050 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:54.831616 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.832023 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.832026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.832027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.832102 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:54.832377 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:54.832380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:54.832382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:54.832600 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:54.832705 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.832924 (+ 219us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.833025 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:15:54.836757 31802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:54.840061 31802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:54.856686 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.849557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53334 (local address 127.23.88.67:36601)
0805 23:15:54.850656 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.850671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.850700 (+ 29us) server_negotiation.cc:407] Connection header received
0805 23:15:54.851067 (+ 367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.851105 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.851353 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.851532 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:54.851984 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.853317 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.855152 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.855757 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.855898 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.856092 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.856373 (+ 281us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.856496 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":141,"threads_started":1}
I20260805 23:15:54.856703 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.850141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.0.0.1:48750)
0805 23:15:54.850977 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.850991 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.851185 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.851840 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.851843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.851861 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:54.851957 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.851966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.853479 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.853483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.854644 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.854655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.856153 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.856157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.856283 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.856325 (+ 42us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.856392 (+ 67us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.856529 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":102,"thread_start_us":97,"threads_started":1}
I20260805 23:15:54.856765 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.850616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.23.88.65:37881)
0805 23:15:54.850935 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.850942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.851136 (+ 194us) server_negotiation.cc:407] Connection header received
0805 23:15:54.851365 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.851370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.851557 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.851694 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:54.852102 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.853348 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.854798 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.855789 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.855927 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.856088 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.856495 (+ 407us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.856610 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260805 23:15:54.857501 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.850312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.23.88.66:40531)
0805 23:15:54.850990 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.851001 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.851026 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:15:54.851138 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.851146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.851375 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.851754 (+ 379us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:54.851925 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.852983 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.854942 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.855793 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.857024 (+ 1231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.857172 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.857219 (+ 47us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.857347 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":132,"threads_started":1}
I20260805 23:15:54.857363 32335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.849342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:36601 (local address 127.0.0.1:53334)
0805 23:15:54.849886 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.849948 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.850091 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.851535 (+ 1444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.851541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.851565 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:54.851825 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.851836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.853479 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.853483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.854992 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.855006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.855911 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.855915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.856053 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.856211 (+ 158us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.856331 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.856506 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":347,"thread_start_us":140,"threads_started":1}
I20260805 23:15:54.857515 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.849791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:40531 (local address 127.0.0.1:47656)
0805 23:15:54.850363 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.850378 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.850474 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.851553 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.851556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.851577 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:54.851825 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.851836 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.853112 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.853118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.854800 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.854813 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.855955 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.855959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.856143 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.856192 (+ 49us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.856300 (+ 108us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.857445 (+ 1145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":245,"thread_start_us":111,"threads_started":1}
I20260805 23:15:54.860836 31948 tablet_service.cc:1467] Processing CreateTablet for tablet 5d05a7158b324925944ef93b3fc11e10 (DEFAULT_TABLE table=test-table [id=40b68b31568945db9be63a68febf6d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:54.860872 32242 tablet_service.cc:1467] Processing CreateTablet for tablet 5d05a7158b324925944ef93b3fc11e10 (DEFAULT_TABLE table=test-table [id=40b68b31568945db9be63a68febf6d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:54.861552 32095 tablet_service.cc:1467] Processing CreateTablet for tablet 5d05a7158b324925944ef93b3fc11e10 (DEFAULT_TABLE table=test-table [id=40b68b31568945db9be63a68febf6d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:54.862170 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d05a7158b324925944ef93b3fc11e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:54.862171 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d05a7158b324925944ef93b3fc11e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:54.862509 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d05a7158b324925944ef93b3fc11e10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:54.869959 32342 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Bootstrap starting.
I20260805 23:15:54.871510 32343 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Bootstrap starting.
I20260805 23:15:54.873363 32342 tablet_bootstrap.cc:654] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:54.873621 32343 tablet_bootstrap.cc:654] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:54.874320 32343 log.cc:824] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:54.874323 32342 log.cc:824] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:54.875166 32344 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Bootstrap starting.
I20260805 23:15:54.876482 32342 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: No bootstrap required, opened a new log
I20260805 23:15:54.876717 32342 ts_tablet_manager.cc:1397] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:15:54.877089 32344 tablet_bootstrap.cc:654] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:54.877799 32344 log.cc:824] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:54.879746 32344 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: No bootstrap required, opened a new log
I20260805 23:15:54.879942 32344 ts_tablet_manager.cc:1397] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260805 23:15:54.880621 32343 tablet_bootstrap.cc:492] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: No bootstrap required, opened a new log
I20260805 23:15:54.880862 32343 ts_tablet_manager.cc:1397] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260805 23:15:54.881160 32342 raft_consensus.cc:348] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.881481 32342 raft_consensus.cc:374] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:54.881548 32342 raft_consensus.cc:729] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Initialized, Role: FOLLOWER
I20260805 23:15:54.882056 32342 consensus_queue.cc:260] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.882804 32015 heartbeater.cc:503] Master 127.23.88.126:37819 was elected leader, sending a full tablet report...
I20260805 23:15:54.883224 32342 ts_tablet_manager.cc:1428] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260805 23:15:54.883734 32344 raft_consensus.cc:348] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.884034 32344 raft_consensus.cc:374] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:54.884101 32344 raft_consensus.cc:729] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Initialized, Role: FOLLOWER
I20260805 23:15:54.884557 32344 consensus_queue.cc:260] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.885322 32313 heartbeater.cc:503] Master 127.23.88.126:37819 was elected leader, sending a full tablet report...
I20260805 23:15:54.885651 32344 ts_tablet_manager.cc:1428] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260805 23:15:54.884986 32343 raft_consensus.cc:348] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.886168 32343 raft_consensus.cc:374] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:54.886266 32343 raft_consensus.cc:729] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Initialized, Role: FOLLOWER
I20260805 23:15:54.886678 32343 consensus_queue.cc:260] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.887693 32343 ts_tablet_manager.cc:1428] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:15:54.888024 32167 heartbeater.cc:503] Master 127.23.88.126:37819 was elected leader, sending a full tablet report...
I20260805 23:15:54.934468 32349 raft_consensus.cc:482] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:54.934680 32349 raft_consensus.cc:504] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.935912 32349 leader_election.cc:288] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531)
I20260805 23:15:54.939864 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.936168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.23.88.67:32945)
0805 23:15:54.936433 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.936449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.936564 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.936883 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.936886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.936903 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:54.937025 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.937031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.937819 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.937823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.938640 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.938650 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.939401 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.939405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.939517 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.939535 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.939626 (+ 91us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.939714 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1}
I20260805 23:15:54.939853 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.936307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:32945 (local address 127.23.88.65:37881)
0805 23:15:54.936458 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.936463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.936539 (+ 76us) server_negotiation.cc:407] Connection header received
0805 23:15:54.936733 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.936735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.936781 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.936876 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:54.937168 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.937696 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.938768 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.939267 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.939401 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.939462 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.939643 (+ 181us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.939716 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:15:54.940296 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.936194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:40531 (local address 127.23.88.67:40669)
0805 23:15:54.936639 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:54.936650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:54.936781 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:54.937045 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:54.937048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:54.937064 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:54.937198 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.937203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.937940 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.937947 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.939157 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:54.939166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.939822 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.939825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:54.939932 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.939962 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:15:54.940085 (+ 123us) client_negotiation.cc:240] Negotiation successful
0805 23:15:54.940150 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":23,"thread_start_us":147,"threads_started":1}
I20260805 23:15:54.940333 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.936350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:40669 (local address 127.23.88.66:40531)
0805 23:15:54.936486 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.936493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.936768 (+ 275us) server_negotiation.cc:407] Connection header received
0805 23:15:54.936882 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.936885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.936940 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.937032 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:54.937317 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.937816 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.939292 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.939694 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.939825 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.939875 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.940121 (+ 246us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.940205 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:15:54.940613 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d05a7158b324925944ef93b3fc11e10" candidate_uuid: "7d6de65e1e71410bb07f0b82f068c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d" is_pre_election: true
I20260805 23:15:54.941012 31968 raft_consensus.cc:2454] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6de65e1e71410bb07f0b82f068c088 in term 0.
I20260805 23:15:54.941007 32117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d05a7158b324925944ef93b3fc11e10" candidate_uuid: "7d6de65e1e71410bb07f0b82f068c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821" is_pre_election: true
I20260805 23:15:54.941285 32117 raft_consensus.cc:2454] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6de65e1e71410bb07f0b82f068c088 in term 0.
I20260805 23:15:54.941610 32192 leader_election.cc:302] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de65e1e71410bb07f0b82f068c088, cc45e232208949ccb94739a0f9974e6d; no voters:
I20260805 23:15:54.941901 32349 raft_consensus.cc:2781] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:54.941980 32349 raft_consensus.cc:482] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:54.942031 32349 raft_consensus.cc:3037] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:54.943130 32349 raft_consensus.cc:504] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.943529 32349 leader_election.cc:288] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [CANDIDATE]: Term 1 election: Requested vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531)
I20260805 23:15:54.943888 32117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d05a7158b324925944ef93b3fc11e10" candidate_uuid: "7d6de65e1e71410bb07f0b82f068c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821"
I20260805 23:15:54.943928 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d05a7158b324925944ef93b3fc11e10" candidate_uuid: "7d6de65e1e71410bb07f0b82f068c088" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d"
I20260805 23:15:54.944037 32117 raft_consensus.cc:3037] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:54.944088 31968 raft_consensus.cc:3037] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:54.945230 31968 raft_consensus.cc:2454] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6de65e1e71410bb07f0b82f068c088 in term 1.
I20260805 23:15:54.945554 32117 raft_consensus.cc:2454] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6de65e1e71410bb07f0b82f068c088 in term 1.
I20260805 23:15:54.945590 32192 leader_election.cc:302] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de65e1e71410bb07f0b82f068c088, cc45e232208949ccb94739a0f9974e6d; no voters:
I20260805 23:15:54.945765 32349 raft_consensus.cc:2781] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:54.946031 32349 raft_consensus.cc:686] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Running, Role: LEADER
I20260805 23:15:54.946379 32349 consensus_queue.cc:237] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } }
I20260805 23:15:54.949800 31801 catalog_manager.cc:5495] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "7d6de65e1e71410bb07f0b82f068c088" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:54.982169 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:54.978611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53346 (local address 127.23.88.67:36601)
0805 23:15:54.978749 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:54.978752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:54.978767 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:15:54.978853 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:54.978855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:54.978911 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:54.979007 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:54.979259 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.979760 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.980632 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:54.980907 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:54.981027 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:54.981278 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:54.981794 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:54.981893 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:54.981985 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 23:15:54.982044 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:15:54.988752 32117 raft_consensus.cc:1261] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Refusing update from remote peer 7d6de65e1e71410bb07f0b82f068c088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:54.988752 31968 raft_consensus.cc:1261] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Refusing update from remote peer 7d6de65e1e71410bb07f0b82f068c088: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:54.989540 32352 consensus_queue.cc:1035] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:54.989746 32349 consensus_queue.cc:1035] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:54.998155 32355 mvcc.cc:205] Tried to move back new op lower bound from 7315340308425347072 to 7315340308263079936. Current Snapshot: MvccSnapshot[applied={T|T < 7315340308425347072}]
I20260805 23:15:55.000771 32357 mvcc.cc:205] Tried to move back new op lower bound from 7315340308425347072 to 7315340308263079936. Current Snapshot: MvccSnapshot[applied={T|T < 7315340308425347072 or (T in {7315340308425347072})}]
I20260805 23:15:55.002018 32356 mvcc.cc:205] Tried to move back new op lower bound from 7315340308425347072 to 7315340308263079936. Current Snapshot: MvccSnapshot[applied={T|T < 7315340308425347072}]
I20260805 23:15:55.007206 31801 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.007403 31801 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.010087 31801 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Sending DeleteTablet for 3 replicas of tablet 5d05a7158b324925944ef93b3fc11e10
I20260805 23:15:55.011134 32095 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d05a7158b324925944ef93b3fc11e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:47656
I20260805 23:15:55.011415 31948 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d05a7158b324925944ef93b3fc11e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260805 23:15:55.011778 32364 tablet_replica.cc:330] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: stopping tablet replica
I20260805 23:15:55.011868 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d05a7158b324925944ef93b3fc11e10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:53334
I20260805 23:15:55.012260 32364 raft_consensus.cc:2229] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.012192 31801 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260805 23:15:55.012348 32366 tablet_replica.cc:330] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: stopping tablet replica
I20260805 23:15:55.012609 32366 raft_consensus.cc:2229] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 LEADER]: Raft consensus shutting down.
W20260805 23:15:55.012643 31801 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:55.012917 32364 raft_consensus.cc:2258] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.012990 32366 raft_consensus.cc:2258] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.014611 32366 ts_tablet_manager.cc:1905] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.014611 32364 ts_tablet_manager.cc:1905] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.017489 32364 ts_tablet_manager.cc:1918] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.017678 32364 log.cc:1196] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/5d05a7158b324925944ef93b3fc11e10
I20260805 23:15:55.017968 31948 tablet_service.cc:1467] Processing CreateTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d (DEFAULT_TABLE table=test-table [id=558386cdb714437ebffbda233beab49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.018193 32364 ts_tablet_manager.cc:1939] T 5d05a7158b324925944ef93b3fc11e10 P d61ee8e651624026bd77bab9fd78d821: Deleting consensus metadata
I20260805 23:15:55.018213 32095 tablet_service.cc:1467] Processing CreateTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d (DEFAULT_TABLE table=test-table [id=558386cdb714437ebffbda233beab49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.018271 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755aa8ca7ef4bfb823e726982d2687d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.018463 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755aa8ca7ef4bfb823e726982d2687d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.019734 31788 catalog_manager.cc:4841] TS d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531): tablet 5d05a7158b324925944ef93b3fc11e10 (table test-table [id=40b68b31568945db9be63a68febf6d85]) successfully deleted
I20260805 23:15:55.020673 32343 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Bootstrap starting.
I20260805 23:15:55.020799 32342 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Bootstrap starting.
I20260805 23:15:55.021067 32366 ts_tablet_manager.cc:1918] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.021173 32366 log.cc:1196] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/5d05a7158b324925944ef93b3fc11e10
I20260805 23:15:55.021498 32366 ts_tablet_manager.cc:1939] T 5d05a7158b324925944ef93b3fc11e10 P 7d6de65e1e71410bb07f0b82f068c088: Deleting consensus metadata
I20260805 23:15:55.021782 32342 tablet_bootstrap.cc:654] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.021781 32343 tablet_bootstrap.cc:654] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.023218 31787 catalog_manager.cc:4841] TS 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601): tablet 5d05a7158b324925944ef93b3fc11e10 (table test-table [id=40b68b31568945db9be63a68febf6d85]) successfully deleted
I20260805 23:15:55.023439 32343 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: No bootstrap required, opened a new log
I20260805 23:15:55.023531 32343 ts_tablet_manager.cc:1397] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:55.024008 32343 raft_consensus.cc:348] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.024127 32343 raft_consensus.cc:374] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.024156 32343 raft_consensus.cc:729] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.024269 32343 consensus_queue.cc:260] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.024412 32342 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: No bootstrap required, opened a new log
I20260805 23:15:55.024518 32342 ts_tablet_manager.cc:1397] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:55.024544 32343 ts_tablet_manager.cc:1428] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.024933 32242 tablet_service.cc:1467] Processing CreateTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d (DEFAULT_TABLE table=test-table [id=558386cdb714437ebffbda233beab49b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.025012 32342 raft_consensus.cc:348] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.025143 32342 raft_consensus.cc:374] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.025177 32342 raft_consensus.cc:729] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.025211 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755aa8ca7ef4bfb823e726982d2687d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.025301 32342 consensus_queue.cc:260] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.025593 32342 ts_tablet_manager.cc:1428] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260805 23:15:55.026002 32365 tablet_replica.cc:330] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: stopping tablet replica
I20260805 23:15:55.026440 32365 raft_consensus.cc:2229] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.026834 32365 raft_consensus.cc:2258] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.028848 32365 ts_tablet_manager.cc:1905] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.029497 32344 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Bootstrap starting.
I20260805 23:15:55.030519 32344 tablet_bootstrap.cc:654] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.031864 32344 tablet_bootstrap.cc:492] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: No bootstrap required, opened a new log
I20260805 23:15:55.031971 32344 ts_tablet_manager.cc:1397] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260805 23:15:55.032294 32365 ts_tablet_manager.cc:1918] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.032407 32365 log.cc:1196] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/5d05a7158b324925944ef93b3fc11e10
I20260805 23:15:55.032545 32344 raft_consensus.cc:348] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.032697 32344 raft_consensus.cc:374] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.032805 32365 ts_tablet_manager.cc:1939] T 5d05a7158b324925944ef93b3fc11e10 P cc45e232208949ccb94739a0f9974e6d: Deleting consensus metadata
I20260805 23:15:55.032938 32344 raft_consensus.cc:729] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.033139 32344 consensus_queue.cc:260] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.034072 32344 ts_tablet_manager.cc:1428] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260805 23:15:55.034323 31787 catalog_manager.cc:4841] TS cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881): tablet 5d05a7158b324925944ef93b3fc11e10 (table test-table [id=40b68b31568945db9be63a68febf6d85]) successfully deleted
W20260805 23:15:55.038919 32314 tablet.cc:2369] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:55.061960 32168 tablet.cc:2369] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:15:55.082713 32017 tablet.cc:2369] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:15:55.285928 32350 raft_consensus.cc:482] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:55.286123 32350 raft_consensus.cc:504] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.287364 32350 leader_election.cc:288] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.292268 32370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.287721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.23.88.66:49443)
0805 23:15:55.288036 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.288058 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.288211 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.288652 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.288655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.288682 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:55.288827 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.288834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.289697 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.289701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.291116 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.291127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.291911 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.291915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.291995 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.292017 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.292078 (+ 61us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.292139 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":58,"thread_start_us":102,"threads_started":1}
I20260805 23:15:55.292330 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.287831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:49443 (local address 127.23.88.65:37881)
0805 23:15:55.288005 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.288011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.288145 (+ 134us) server_negotiation.cc:407] Connection header received
0805 23:15:55.288369 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.288374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.288436 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.288492 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:55.288986 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.289574 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.291244 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.291781 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.291903 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.291961 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.292119 (+ 158us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.292198 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:15:55.292881 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755aa8ca7ef4bfb823e726982d2687d" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d" is_pre_election: true
I20260805 23:15:55.293035 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.288354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:40953 (local address 127.23.88.67:36601)
0805 23:15:55.288515 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.288519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.288583 (+ 64us) server_negotiation.cc:407] Connection header received
0805 23:15:55.288767 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.288769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.288824 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.288911 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:55.289190 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.289781 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.291719 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.292318 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.292443 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.292497 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.292846 (+ 349us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.292921 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:15:55.293057 32371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.288215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:36601 (local address 127.23.88.66:40953)
0805 23:15:55.288516 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.288529 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.288650 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.288944 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.288947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.288971 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:55.289078 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.289084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.289884 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.289887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.291116 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.291127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.292564 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.292568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.292666 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.292692 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.292823 (+ 131us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.292899 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":56,"thread_start_us":86,"threads_started":1}
I20260805 23:15:55.293054 31968 raft_consensus.cc:2454] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 0.
I20260805 23:15:55.293474 32043 leader_election.cc:302] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc45e232208949ccb94739a0f9974e6d, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.293614 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755aa8ca7ef4bfb823e726982d2687d" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088" is_pre_election: true
I20260805 23:15:55.293774 32350 raft_consensus.cc:2781] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:55.293829 32350 raft_consensus.cc:482] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:55.293877 32350 raft_consensus.cc:3037] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.293889 32264 raft_consensus.cc:2454] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 0.
I20260805 23:15:55.294989 32350 raft_consensus.cc:504] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.295498 32350 leader_election.cc:288] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 election: Requested vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.295837 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755aa8ca7ef4bfb823e726982d2687d" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d"
I20260805 23:15:55.295885 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755aa8ca7ef4bfb823e726982d2687d" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088"
I20260805 23:15:55.295991 32264 raft_consensus.cc:3037] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.295986 31968 raft_consensus.cc:3037] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.296969 32264 raft_consensus.cc:2454] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 1.
I20260805 23:15:55.297201 31968 raft_consensus.cc:2454] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 1.
I20260805 23:15:55.297290 32043 leader_election.cc:302] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de65e1e71410bb07f0b82f068c088, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.297457 32350 raft_consensus.cc:2781] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:55.297634 32350 raft_consensus.cc:686] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 1 LEADER]: Becoming Leader. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Running, Role: LEADER
I20260805 23:15:55.297942 32350 consensus_queue.cc:237] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.300431 31802 catalog_manager.cc:5495] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 reported cstate change: term changed from 0 to 1, leader changed from <none> to d61ee8e651624026bd77bab9fd78d821 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "d61ee8e651624026bd77bab9fd78d821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971753, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.126@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:55.362051 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.352926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37819 (local address 127.0.0.1:55840)
0805 23:15:55.353364 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.353377 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.353479 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.353733 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.353735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.354126 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:55.354225 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.354231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.354977 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.354979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.355608 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.355617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.356039 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.356042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.356102 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.356630 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:55.356642 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:55.358235 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:55.360272 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:55.360281 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:55.360286 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:55.360564 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:55.360811 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:55.360813 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:55.360814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:55.360890 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:55.361392 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:55.361402 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:55.361553 (+ 151us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.361685 (+ 132us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.361835 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":132,"threads_started":1}
I20260805 23:15:55.362078 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.352964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55840 (local address 127.23.88.126:37819)
0805 23:15:55.353245 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.353251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.353460 (+ 209us) server_negotiation.cc:407] Connection header received
0805 23:15:55.353563 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.353566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.353607 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.353693 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:55.354333 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.354871 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.355711 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.355938 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.356017 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.358366 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:55.358369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:55.358371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:55.358393 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:55.360174 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:55.360650 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:55.360653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:55.360654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:55.360731 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:55.360967 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:55.360969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:55.360970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:55.361192 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:55.361293 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.361781 (+ 488us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.361906 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":114,"threads_started":1}
I20260805 23:15:55.367050 31802 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55840:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260805 23:15:55.367985 31802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:55.373581 32095 tablet_service.cc:1467] Processing CreateTablet for tablet adf1b4d07db94ef78371f3c7160e2733 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5537a7a0f1fc421ca5521ccf646504b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:55.373581 32242 tablet_service.cc:1467] Processing CreateTablet for tablet adf1b4d07db94ef78371f3c7160e2733 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5537a7a0f1fc421ca5521ccf646504b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:55.373566 31948 tablet_service.cc:1467] Processing CreateTablet for tablet adf1b4d07db94ef78371f3c7160e2733 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5537a7a0f1fc421ca5521ccf646504b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260805 23:15:55.374010 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf1b4d07db94ef78371f3c7160e2733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.374010 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf1b4d07db94ef78371f3c7160e2733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.374010 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adf1b4d07db94ef78371f3c7160e2733. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.376848 32344 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: Bootstrap starting.
I20260805 23:15:55.376864 32342 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: Bootstrap starting.
I20260805 23:15:55.377754 32344 tablet_bootstrap.cc:654] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.377807 32342 tablet_bootstrap.cc:654] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.378607 32343 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: Bootstrap starting.
I20260805 23:15:55.379047 32344 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: No bootstrap required, opened a new log
I20260805 23:15:55.379184 32344 ts_tablet_manager.cc:1397] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260805 23:15:55.379297 32342 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: No bootstrap required, opened a new log
I20260805 23:15:55.379390 32342 ts_tablet_manager.cc:1397] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:55.379693 32343 tablet_bootstrap.cc:654] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.379683 32344 raft_consensus.cc:348] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.379810 32344 raft_consensus.cc:374] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.379840 32344 raft_consensus.cc:729] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.379873 32342 raft_consensus.cc:348] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.379984 32342 raft_consensus.cc:374] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.379956 32344 consensus_queue.cc:260] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.380033 32342 raft_consensus.cc:729] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.380223 32344 ts_tablet_manager.cc:1428] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:15:55.380194 32342 consensus_queue.cc:260] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.380405 32352 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.380553 32352 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.380607 32342 ts_tablet_manager.cc:1428] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.380798 32348 raft_consensus.cc:482] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:55.380899 32348 raft_consensus.cc:504] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.381404 32385 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.381553 32385 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.382220 32348 leader_election.cc:288] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.382529 32343 tablet_bootstrap.cc:492] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: No bootstrap required, opened a new log
I20260805 23:15:55.382619 32343 ts_tablet_manager.cc:1397] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260805 23:15:55.383070 32343 raft_consensus.cc:348] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.383252 32343 raft_consensus.cc:374] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.383294 32343 raft_consensus.cc:729] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.383445 32343 consensus_queue.cc:260] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.383749 32343 ts_tablet_manager.cc:1428] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.383980 32350 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.384147 32350 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.387698 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.382552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:53881 (local address 127.23.88.66:40531)
0805 23:15:55.382672 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.382676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.383498 (+ 822us) server_negotiation.cc:407] Connection header received
0805 23:15:55.383711 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.383714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.383767 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.383844 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:55.384377 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.385109 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.386680 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.387178 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.387285 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.387326 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.387548 (+ 222us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.387608 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:15:55.387751 32388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.382513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:40531 (local address 127.23.88.65:53881)
0805 23:15:55.383408 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.383424 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.383569 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.384006 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.384008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.384021 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:55.384233 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.384245 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.385279 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.385283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.386558 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.386567 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.387301 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.387305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.387420 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.387447 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.387526 (+ 79us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.387591 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"thread_start_us":112,"threads_started":1}
I20260805 23:15:55.388195 32116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf1b4d07db94ef78371f3c7160e2733" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821" is_pre_election: true
I20260805 23:15:55.388339 32116 raft_consensus.cc:2454] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 0.
I20260805 23:15:55.388759 31896 leader_election.cc:302] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc45e232208949ccb94739a0f9974e6d, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.389055 32348 raft_consensus.cc:2781] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:55.389123 32348 raft_consensus.cc:482] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:55.389179 32348 raft_consensus.cc:3037] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.389217 32387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.382966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:36601 (local address 127.23.88.65:42949)
0805 23:15:55.383297 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.383313 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.383424 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.385771 (+ 2347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.385774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.385792 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:15:55.385928 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.385933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.386725 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.386727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.387709 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.387718 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.388792 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.388795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.388890 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.388915 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.388999 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.389074 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
I20260805 23:15:55.389241 32337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.385442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:42949 (local address 127.23.88.67:36601)
0805 23:15:55.385554 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.385558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.385571 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:15:55.385616 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.385619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.385659 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.385758 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:15:55.386047 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.386643 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.388109 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.388671 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.388768 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.388804 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.389021 (+ 217us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.389104 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260805 23:15:55.389653 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf1b4d07db94ef78371f3c7160e2733" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088" is_pre_election: true
I20260805 23:15:55.389767 32264 raft_consensus.cc:2454] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 0.
I20260805 23:15:55.390784 32348 raft_consensus.cc:504] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.391224 32348 leader_election.cc:288] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 election: Requested vote from peers d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.391606 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf1b4d07db94ef78371f3c7160e2733" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088"
I20260805 23:15:55.391594 32116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adf1b4d07db94ef78371f3c7160e2733" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821"
I20260805 23:15:55.391722 32116 raft_consensus.cc:3037] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.391723 32264 raft_consensus.cc:3037] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.392956 32264 raft_consensus.cc:2454] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 1.
I20260805 23:15:55.392949 32116 raft_consensus.cc:2454] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 1.
I20260805 23:15:55.393241 31896 leader_election.cc:302] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc45e232208949ccb94739a0f9974e6d, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.393411 32348 raft_consensus.cc:2781] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:55.393515 32348 raft_consensus.cc:686] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [term 1 LEADER]: Becoming Leader. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Running, Role: LEADER
I20260805 23:15:55.393780 32348 consensus_queue.cc:237] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.395004 32385 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: TxnStatusTablet state changed. Reason: New leader cc45e232208949ccb94739a0f9974e6d. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.395134 32385 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:55.395565 32390 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260805 23:15:55.395723 32390 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260805 23:15:55.396328 31801 catalog_manager.cc:5495] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d reported cstate change: term changed from 0 to 1, leader changed from <none> to cc45e232208949ccb94739a0f9974e6d (127.23.88.65). New cstate: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:55.409886 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.405154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48758 (local address 127.23.88.65:37881)
0805 23:15:55.405302 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.405306 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.405478 (+ 172us) server_negotiation.cc:407] Connection header received
0805 23:15:55.405760 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.405762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.405827 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.405920 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.406228 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.406754 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.408111 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.408342 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.408458 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.408894 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.409371 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.409493 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.409685 (+ 192us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.409755 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:15:55.409901 32380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.405062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.0.0.1:48758)
0805 23:15:55.405343 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.405368 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.405550 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.405968 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.405972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.405981 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:55.406110 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.406117 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.406876 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.406879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.407951 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.407966 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.408501 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.408507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.408646 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.408719 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.409508 (+ 789us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.409530 (+ 22us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.409612 (+ 82us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.409729 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168}
I20260805 23:15:55.413535 32116 raft_consensus.cc:1261] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Refusing update from remote peer cc45e232208949ccb94739a0f9974e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.413692 32264 raft_consensus.cc:1261] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Refusing update from remote peer cc45e232208949ccb94739a0f9974e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.414147 32348 consensus_queue.cc:1035] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.414359 32385 consensus_queue.cc:1035] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.416250 32350 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: TxnStatusTablet state changed. Reason: New leader cc45e232208949ccb94739a0f9974e6d. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.416386 32350 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.417192 32352 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: TxnStatusTablet state changed. Reason: New leader cc45e232208949ccb94739a0f9974e6d. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.417327 32352 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.418776 32385 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.418920 32385 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:55.418783 32348 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.419220 32348 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P cc45e232208949ccb94739a0f9974e6d: This TxnStatusTablet replica's current role is: LEADER
I20260805 23:15:55.420648 32352 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.420790 32352 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P 7d6de65e1e71410bb07f0b82f068c088: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.421264 32350 tablet_replica.cc:439] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } } }
I20260805 23:15:55.421380 32350 tablet_replica.cc:442] T adf1b4d07db94ef78371f3c7160e2733 P d61ee8e651624026bd77bab9fd78d821: This TxnStatusTablet replica's current role is: FOLLOWER
I20260805 23:15:55.430156 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.426006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47668 (local address 127.23.88.66:40531)
0805 23:15:55.426141 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.426146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.426189 (+ 43us) server_negotiation.cc:407] Connection header received
0805 23:15:55.426406 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.426410 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.426480 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.426605 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.426867 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.427558 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.428559 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.428817 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.428939 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.429247 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.429716 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.429850 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.429933 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.429993 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:15:55.442605 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.438323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:34841 (local address 127.23.88.65:37881)
0805 23:15:55.438541 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.438545 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.438703 (+ 158us) server_negotiation.cc:407] Connection header received
0805 23:15:55.438833 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.438836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.438885 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.438957 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.439655 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.440222 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.441520 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.441740 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.441823 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.442181 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.442256 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.442309 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.442435 (+ 126us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.442485 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260805 23:15:55.442691 32398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.438216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.23.88.66:34841)
0805 23:15:55.438587 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.438604 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.438725 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.438990 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.438993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.439002 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:55.439517 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.439527 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.440357 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.440360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.441404 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.441417 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.441880 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.441884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.441980 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.442043 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.442336 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.442342 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.442411 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.442502 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":101,"threads_started":1}
I20260805 23:15:55.450762 32264 raft_consensus.cc:1261] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Refusing update from remote peer d61ee8e651624026bd77bab9fd78d821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.450845 31968 raft_consensus.cc:1261] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Refusing update from remote peer d61ee8e651624026bd77bab9fd78d821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.451336 32350 consensus_queue.cc:1035] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.452286 32350 consensus_queue.cc:1035] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.476619 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.472261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:50755 (local address 127.23.88.66:40531)
0805 23:15:55.472371 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.472375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.472719 (+ 344us) server_negotiation.cc:407] Connection header received
0805 23:15:55.472912 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.472916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.472982 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.473029 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.473439 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.473941 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.475335 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.475593 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.475678 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.476106 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.476190 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.476270 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.476411 (+ 141us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.476473 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:15:55.477039 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.472182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:40531 (local address 127.23.88.65:50755)
0805 23:15:55.472638 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.472653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.472796 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.473174 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.473177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.473184 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:55.473322 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.473328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.474048 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.474051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.475161 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.475174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.475777 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.475782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.475891 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.475962 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.476288 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.476296 (+ 8us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.476781 (+ 485us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.476871 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":123,"threads_started":1}
I20260805 23:15:55.500667 31802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.500887 31802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.503248 31802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Sending DeleteTablet for 3 replicas of tablet 2755aa8ca7ef4bfb823e726982d2687d
I20260805 23:15:55.503877 31948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260805 23:15:55.503861 32095 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:47656
I20260805 23:15:55.504156 32364 tablet_replica.cc:330] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: stopping tablet replica
I20260805 23:15:55.504295 32364 raft_consensus.cc:2229] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:55.504491 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755aa8ca7ef4bfb823e726982d2687d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:53334
I20260805 23:15:55.504647 32364 raft_consensus.cc:2258] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.504705 32366 tablet_replica.cc:330] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: stopping tablet replica
I20260805 23:15:55.504858 32366 raft_consensus.cc:2229] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.504990 32366 raft_consensus.cc:2258] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.505818 32364 ts_tablet_manager.cc:1905] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.505818 32366 ts_tablet_manager.cc:1905] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.508852 32364 ts_tablet_manager.cc:1918] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:55.508852 32366 ts_tablet_manager.cc:1918] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:55.508935 32364 log.cc:1196] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/2755aa8ca7ef4bfb823e726982d2687d
I20260805 23:15:55.508941 32366 log.cc:1196] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/2755aa8ca7ef4bfb823e726982d2687d
I20260805 23:15:55.509207 32366 ts_tablet_manager.cc:1939] T 2755aa8ca7ef4bfb823e726982d2687d P 7d6de65e1e71410bb07f0b82f068c088: Deleting consensus metadata
I20260805 23:15:55.509207 32364 ts_tablet_manager.cc:1939] T 2755aa8ca7ef4bfb823e726982d2687d P d61ee8e651624026bd77bab9fd78d821: Deleting consensus metadata
I20260805 23:15:55.510270 31787 catalog_manager.cc:4841] TS 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601): tablet 2755aa8ca7ef4bfb823e726982d2687d (table test-table [id=558386cdb714437ebffbda233beab49b]) successfully deleted
I20260805 23:15:55.510499 31788 catalog_manager.cc:4841] TS d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531): tablet 2755aa8ca7ef4bfb823e726982d2687d (table test-table [id=558386cdb714437ebffbda233beab49b]) successfully deleted
I20260805 23:15:55.514007 32365 tablet_replica.cc:330] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: stopping tablet replica
I20260805 23:15:55.514192 32365 raft_consensus.cc:2229] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.514374 32365 raft_consensus.cc:2258] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.515432 32365 ts_tablet_manager.cc:1905] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.519655 32365 ts_tablet_manager.cc:1918] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:55.519778 32365 log.cc:1196] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/2755aa8ca7ef4bfb823e726982d2687d
I20260805 23:15:55.520114 32365 ts_tablet_manager.cc:1939] T 2755aa8ca7ef4bfb823e726982d2687d P cc45e232208949ccb94739a0f9974e6d: Deleting consensus metadata
I20260805 23:15:55.521085 31787 catalog_manager.cc:4841] TS cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881): tablet 2755aa8ca7ef4bfb823e726982d2687d (table test-table [id=558386cdb714437ebffbda233beab49b]) successfully deleted
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[31756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:15:55.540346 32381 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:55.532916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55850 (local address 127.23.88.126:37819)
0805 23:15:55.533051 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.533055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.533395 (+ 340us) server_negotiation.cc:407] Connection header received
0805 23:15:55.533587 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.533590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.533647 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.533731 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:55.534469 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.535028 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.536320 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.536745 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.536855 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.540199 (+ 3344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55850: BlockingRecv error: recv got EOF from 127.0.0.1:55850 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[31756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM
I20260805 23:15:55.551651 32381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.542519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55854 (local address 127.23.88.126:37819)
0805 23:15:55.542643 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.542648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.543059 (+ 411us) server_negotiation.cc:407] Connection header received
0805 23:15:55.543243 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.543246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.543300 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.543390 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:55.544039 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.544569 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.545378 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.545612 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.545721 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.548301 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:55.548305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:55.548308 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:55.548345 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:55.549856 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:55.550302 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:55.550306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:55.550308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:55.550397 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:55.550695 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:55.550699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:55.550701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:55.550905 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:55.551055 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.551337 (+ 282us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.551477 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:15:55.554723 31802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:55.555276 31802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:55.560530 31948 tablet_service.cc:1467] Processing CreateTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 (DEFAULT_TABLE table=test-table [id=8234977d16e249d7bee23f41860d7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.560730 32095 tablet_service.cc:1467] Processing CreateTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 (DEFAULT_TABLE table=test-table [id=8234977d16e249d7bee23f41860d7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.560886 31948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a5318ee2644fa68c958ed86cfb6219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.561019 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a5318ee2644fa68c958ed86cfb6219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.562461 32242 tablet_service.cc:1467] Processing CreateTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 (DEFAULT_TABLE table=test-table [id=8234977d16e249d7bee23f41860d7140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.562772 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57a5318ee2644fa68c958ed86cfb6219. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.564824 32343 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Bootstrap starting.
I20260805 23:15:55.565420 32344 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Bootstrap starting.
I20260805 23:15:55.565878 32343 tablet_bootstrap.cc:654] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.566308 32344 tablet_bootstrap.cc:654] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.567606 32343 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: No bootstrap required, opened a new log
I20260805 23:15:55.567691 32343 ts_tablet_manager.cc:1397] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:55.568210 32343 raft_consensus.cc:348] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.568326 32343 raft_consensus.cc:374] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.568365 32343 raft_consensus.cc:729] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.568516 32343 consensus_queue.cc:260] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.568773 32343 ts_tablet_manager.cc:1428] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.569532 32342 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Bootstrap starting.
I20260805 23:15:55.570564 32342 tablet_bootstrap.cc:654] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.572636 32342 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: No bootstrap required, opened a new log
I20260805 23:15:55.572759 32342 ts_tablet_manager.cc:1397] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:55.573599 32342 raft_consensus.cc:348] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.573737 32342 raft_consensus.cc:374] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.573781 32342 raft_consensus.cc:729] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.574334 32342 consensus_queue.cc:260] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.574644 32342 ts_tablet_manager.cc:1428] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:55.575948 32344 tablet_bootstrap.cc:492] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: No bootstrap required, opened a new log
I20260805 23:15:55.576079 32344 ts_tablet_manager.cc:1397] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260805 23:15:55.576509 32344 raft_consensus.cc:348] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.576637 32344 raft_consensus.cc:374] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.576678 32344 raft_consensus.cc:729] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.576833 32344 consensus_queue.cc:260] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.577121 32344 ts_tablet_manager.cc:1428] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:15:55.633039 32350 raft_consensus.cc:482] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:55.633221 32350 raft_consensus.cc:504] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.633700 32350 leader_election.cc:288] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.634224 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a5318ee2644fa68c958ed86cfb6219" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d" is_pre_election: true
I20260805 23:15:55.634274 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a5318ee2644fa68c958ed86cfb6219" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088" is_pre_election: true
I20260805 23:15:55.634404 32264 raft_consensus.cc:2454] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 0.
I20260805 23:15:55.634405 31968 raft_consensus.cc:2454] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 0.
I20260805 23:15:55.634753 32043 leader_election.cc:302] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de65e1e71410bb07f0b82f068c088, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.634979 32350 raft_consensus.cc:2781] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:55.635048 32350 raft_consensus.cc:482] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:55.635109 32350 raft_consensus.cc:3037] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.636215 32350 raft_consensus.cc:504] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.636612 32350 leader_election.cc:288] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 election: Requested vote from peers cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.636994 31968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a5318ee2644fa68c958ed86cfb6219" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc45e232208949ccb94739a0f9974e6d"
I20260805 23:15:55.636994 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57a5318ee2644fa68c958ed86cfb6219" candidate_uuid: "d61ee8e651624026bd77bab9fd78d821" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088"
I20260805 23:15:55.637123 32264 raft_consensus.cc:3037] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.637123 31968 raft_consensus.cc:3037] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.638319 32264 raft_consensus.cc:2454] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 1.
I20260805 23:15:55.638319 31968 raft_consensus.cc:2454] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d61ee8e651624026bd77bab9fd78d821 in term 1.
I20260805 23:15:55.638703 32043 leader_election.cc:302] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6de65e1e71410bb07f0b82f068c088, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.639000 32350 raft_consensus.cc:2781] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:55.639135 32350 raft_consensus.cc:686] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 1 LEADER]: Becoming Leader. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Running, Role: LEADER
I20260805 23:15:55.639285 32350 consensus_queue.cc:237] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.641207 31801 catalog_manager.cc:5495] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 reported cstate change: term changed from 0 to 1, leader changed from <none> to d61ee8e651624026bd77bab9fd78d821 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "d61ee8e651624026bd77bab9fd78d821" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:55.659668 32340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.656134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47670 (local address 127.23.88.66:40531)
0805 23:15:55.656286 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.656291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.656307 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:55.656386 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.656389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.656463 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.656544 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.656814 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.657325 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.658422 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.658666 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.658781 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.659121 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.659226 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.659355 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.659453 (+ 98us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.659516 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:15:55.662067 32264 raft_consensus.cc:1261] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Refusing update from remote peer d61ee8e651624026bd77bab9fd78d821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.662191 31968 raft_consensus.cc:1261] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Refusing update from remote peer d61ee8e651624026bd77bab9fd78d821: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.662616 32350 consensus_queue.cc:1035] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.662811 32372 consensus_queue.cc:1035] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.672763 31802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.672963 31802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.675590 31802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Sending DeleteTablet for 3 replicas of tablet 57a5318ee2644fa68c958ed86cfb6219
I20260805 23:15:55.676241 32095 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:47656
I20260805 23:15:55.676462 32364 tablet_replica.cc:330] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: stopping tablet replica
I20260805 23:15:55.676501 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:53334
I20260805 23:15:55.676599 32364 raft_consensus.cc:2229] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:55.676723 32366 tablet_replica.cc:330] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: stopping tablet replica
I20260805 23:15:55.676817 32364 raft_consensus.cc:2258] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.676841 32366 raft_consensus.cc:2229] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.676946 32366 raft_consensus.cc:2258] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.677309 31948 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260805 23:15:55.677438 31946 tablet_service.cc:1514] Processing DeleteTablet for tablet 57a5318ee2644fa68c958ed86cfb6219 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260805 23:15:55.677511 32365 tablet_replica.cc:330] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: stopping tablet replica
I20260805 23:15:55.677625 32365 raft_consensus.cc:2229] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.677760 32365 raft_consensus.cc:2258] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.678256 32364 ts_tablet_manager.cc:1905] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.678280 32366 ts_tablet_manager.cc:1905] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.680074 31802 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:15:55.680541 31802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:15:55.680795 32366 ts_tablet_manager.cc:1918] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.680845 32364 ts_tablet_manager.cc:1918] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.680881 32366 log.cc:1196] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/57a5318ee2644fa68c958ed86cfb6219
I20260805 23:15:55.680923 32364 log.cc:1196] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/57a5318ee2644fa68c958ed86cfb6219
I20260805 23:15:55.681188 32366 ts_tablet_manager.cc:1939] T 57a5318ee2644fa68c958ed86cfb6219 P 7d6de65e1e71410bb07f0b82f068c088: Deleting consensus metadata
I20260805 23:15:55.681227 32364 ts_tablet_manager.cc:1939] T 57a5318ee2644fa68c958ed86cfb6219 P d61ee8e651624026bd77bab9fd78d821: Deleting consensus metadata
I20260805 23:15:55.682027 31788 catalog_manager.cc:4841] TS d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531): tablet 57a5318ee2644fa68c958ed86cfb6219 (table test-table [id=8234977d16e249d7bee23f41860d7140]) successfully deleted
I20260805 23:15:55.682068 31787 catalog_manager.cc:4841] TS 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601): tablet 57a5318ee2644fa68c958ed86cfb6219 (table test-table [id=8234977d16e249d7bee23f41860d7140]) successfully deleted
I20260805 23:15:55.682837 32365 ts_tablet_manager.cc:1905] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.685542 32365 ts_tablet_manager.cc:1918] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:15:55.685642 32365 log.cc:1196] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/57a5318ee2644fa68c958ed86cfb6219
I20260805 23:15:55.685913 32365 ts_tablet_manager.cc:1939] T 57a5318ee2644fa68c958ed86cfb6219 P cc45e232208949ccb94739a0f9974e6d: Deleting consensus metadata
I20260805 23:15:55.686757 31787 catalog_manager.cc:4841] TS cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881): tablet 57a5318ee2644fa68c958ed86cfb6219 (table test-table [id=8234977d16e249d7bee23f41860d7140]) successfully deleted
I20260805 23:15:55.687469 31946 tablet_service.cc:1467] Processing CreateTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 (DEFAULT_TABLE table=test-table [id=5174143e52d94f6badbb2b83c43e34ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.687469 32242 tablet_service.cc:1467] Processing CreateTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 (DEFAULT_TABLE table=test-table [id=5174143e52d94f6badbb2b83c43e34ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.687788 32242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51d4246ebdfa4217a3aea457b2fb1c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.687796 31946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51d4246ebdfa4217a3aea457b2fb1c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:55.690411 32344 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Bootstrap starting.
I20260805 23:15:55.691001 32095 tablet_service.cc:1467] Processing CreateTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 (DEFAULT_TABLE table=test-table [id=5174143e52d94f6badbb2b83c43e34ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:15:55.691354 32344 tablet_bootstrap.cc:654] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.691387 32095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51d4246ebdfa4217a3aea457b2fb1c71. 1 dirs total, 0 dirs full, 0 dirs failed
W20260805 23:15:55.691413 31787 catalog_manager.cc:4808] TS cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881): delete failed for tablet 57a5318ee2644fa68c958ed86cfb6219 because the tablet was not found. No further retry: Not found: Tablet not found: 57a5318ee2644fa68c958ed86cfb6219
I20260805 23:15:55.691911 32342 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: Bootstrap starting.
I20260805 23:15:55.692716 32342 tablet_bootstrap.cc:654] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.693827 32343 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Bootstrap starting.
I20260805 23:15:55.693931 32342 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: No bootstrap required, opened a new log
I20260805 23:15:55.694007 32342 ts_tablet_manager.cc:1397] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260805 23:15:55.694402 32342 raft_consensus.cc:348] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.694516 32342 raft_consensus.cc:374] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.694545 32342 raft_consensus.cc:729] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.694666 32342 consensus_queue.cc:260] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.694909 32342 ts_tablet_manager.cc:1428] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.694916 32343 tablet_bootstrap.cc:654] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:55.695650 32344 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: No bootstrap required, opened a new log
I20260805 23:15:55.695744 32344 ts_tablet_manager.cc:1397] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260805 23:15:55.696252 32344 raft_consensus.cc:348] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.696383 32344 raft_consensus.cc:374] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.696417 32344 raft_consensus.cc:729] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6de65e1e71410bb07f0b82f068c088, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.696547 32344 consensus_queue.cc:260] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.696794 32344 ts_tablet_manager.cc:1428] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:15:55.699339 32343 tablet_bootstrap.cc:492] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: No bootstrap required, opened a new log
I20260805 23:15:55.699471 32343 ts_tablet_manager.cc:1397] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260805 23:15:55.700075 32343 raft_consensus.cc:348] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.700228 32343 raft_consensus.cc:374] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:55.700274 32343 raft_consensus.cc:729] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d61ee8e651624026bd77bab9fd78d821, State: Initialized, Role: FOLLOWER
I20260805 23:15:55.700438 32343 consensus_queue.cc:260] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.701313 32343 ts_tablet_manager.cc:1428] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260805 23:15:55.759382 32395 raft_consensus.cc:482] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:15:55.759621 32395 raft_consensus.cc:504] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.760118 32395 leader_election.cc:288] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.760578 32116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51d4246ebdfa4217a3aea457b2fb1c71" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821" is_pre_election: true
I20260805 23:15:55.760636 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51d4246ebdfa4217a3aea457b2fb1c71" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088" is_pre_election: true
I20260805 23:15:55.760761 32116 raft_consensus.cc:2454] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 0.
I20260805 23:15:55.760876 32264 raft_consensus.cc:2454] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 0.
I20260805 23:15:55.761106 31896 leader_election.cc:302] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc45e232208949ccb94739a0f9974e6d, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.761323 32395 raft_consensus.cc:2781] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:15:55.761390 32395 raft_consensus.cc:482] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:15:55.761466 32395 raft_consensus.cc:3037] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.762544 32395 raft_consensus.cc:504] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.762979 32395 leader_election.cc:288] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 election: Requested vote from peers d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531), 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601)
I20260805 23:15:55.763443 32116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51d4246ebdfa4217a3aea457b2fb1c71" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d61ee8e651624026bd77bab9fd78d821"
I20260805 23:15:55.763447 32264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51d4246ebdfa4217a3aea457b2fb1c71" candidate_uuid: "cc45e232208949ccb94739a0f9974e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6de65e1e71410bb07f0b82f068c088"
I20260805 23:15:55.763561 32116 raft_consensus.cc:3037] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.763588 32264 raft_consensus.cc:3037] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:55.764485 32116 raft_consensus.cc:2454] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 1.
I20260805 23:15:55.764588 32264 raft_consensus.cc:2454] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc45e232208949ccb94739a0f9974e6d in term 1.
I20260805 23:15:55.764781 31896 leader_election.cc:302] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc45e232208949ccb94739a0f9974e6d, d61ee8e651624026bd77bab9fd78d821; no voters:
I20260805 23:15:55.764952 32395 raft_consensus.cc:2781] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:55.765046 32395 raft_consensus.cc:686] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 1 LEADER]: Becoming Leader. State: Replica: cc45e232208949ccb94739a0f9974e6d, State: Running, Role: LEADER
I20260805 23:15:55.765199 32395 consensus_queue.cc:237] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } }
I20260805 23:15:55.767239 31801 catalog_manager.cc:5495] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d reported cstate change: term changed from 0 to 1, leader changed from <none> to cc45e232208949ccb94739a0f9974e6d (127.23.88.65). New cstate: current_term: 1 leader_uuid: "cc45e232208949ccb94739a0f9974e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc45e232208949ccb94739a0f9974e6d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 } health_report { overall_health: UNKNOWN } } }
I20260805 23:15:55.792141 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.785371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48770 (local address 127.23.88.65:37881)
0805 23:15:55.785510 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.785514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.785527 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:15:55.785571 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.785573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.785617 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.785703 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.785938 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.786476 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.790761 (+ 4285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.791175 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.791289 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.791651 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.791746 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.791827 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.791933 (+ 106us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.791995 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:15:55.802435 32341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.799046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:36561 (local address 127.23.88.65:37881)
0805 23:15:55.799232 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:55.799236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:55.799250 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:15:55.799328 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:55.799330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:55.799372 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:55.799449 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:15:55.799688 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.800236 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.801400 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.801621 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.801703 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.801980 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.802050 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.802107 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:55.802195 (+ 88us) server_negotiation.cc:299] Negotiation successful
0805 23:15:55.802269 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":104}
I20260805 23:15:55.802448 32403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.798956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.23.88.65:36561)
0805 23:15:55.799159 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.799173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:55.799250 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:55.799472 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:55.799474 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:55.799481 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0805 23:15:55.799582 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.799588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.800338 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.800341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.801279 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:55.801288 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:55.801731 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:55.801735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:55.801831 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:55.801863 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0805 23:15:55.802126 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0805 23:15:55.802131 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:15:55.802189 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 23:15:55.802292 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":3}
I20260805 23:15:55.808391 32264 raft_consensus.cc:1261] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Refusing update from remote peer cc45e232208949ccb94739a0f9974e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.808853 32116 raft_consensus.cc:1261] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Refusing update from remote peer cc45e232208949ccb94739a0f9974e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:15:55.808893 32385 consensus_queue.cc:1035] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6de65e1e71410bb07f0b82f068c088" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 36601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.809376 32395 consensus_queue.cc:1035] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d61ee8e651624026bd77bab9fd78d821" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:15:55.845628 31802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.845785 31802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:15:55.847983 31802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0b289cd10540404cb73483198827c8f2: Sending DeleteTablet for 3 replicas of tablet 51d4246ebdfa4217a3aea457b2fb1c71
I20260805 23:15:55.848691 31946 tablet_service.cc:1514] Processing DeleteTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260805 23:15:55.848724 32242 tablet_service.cc:1514] Processing DeleteTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:53334
I20260805 23:15:55.848730 32095 tablet_service.cc:1514] Processing DeleteTablet for tablet 51d4246ebdfa4217a3aea457b2fb1c71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:15:55 UTC) from {username='oryx', principal='oryx/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:47656
I20260805 23:15:55.848907 32366 tablet_replica.cc:330] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: stopping tablet replica
I20260805 23:15:55.848898 32365 tablet_replica.cc:330] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: stopping tablet replica
I20260805 23:15:55.848906 32364 tablet_replica.cc:330] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: stopping tablet replica
I20260805 23:15:55.849022 32364 raft_consensus.cc:2229] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.849028 32366 raft_consensus.cc:2229] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:15:55.849030 32365 raft_consensus.cc:2229] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:15:55.849143 32364 raft_consensus.cc:2258] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.849143 32366 raft_consensus.cc:2258] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.849350 32365 raft_consensus.cc:2258] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:15:55.849982 32364 ts_tablet_manager.cc:1905] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.850070 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 31879
I20260805 23:15:55.850464 32365 ts_tablet_manager.cc:1905] T 51d4246ebdfa4217a3aea457b2fb1c71 P cc45e232208949ccb94739a0f9974e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.851648 32366 ts_tablet_manager.cc:1905] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:15:55.854161 32366 ts_tablet_manager.cc:1918] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:55.854259 32366 log.cc:1196] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/51d4246ebdfa4217a3aea457b2fb1c71
I20260805 23:15:55.854241 32364 ts_tablet_manager.cc:1918] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260805 23:15:55.854326 32364 log.cc:1196] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/51d4246ebdfa4217a3aea457b2fb1c71
I20260805 23:15:55.854573 32364 ts_tablet_manager.cc:1939] T 51d4246ebdfa4217a3aea457b2fb1c71 P d61ee8e651624026bd77bab9fd78d821: Deleting consensus metadata
I20260805 23:15:55.854573 32366 ts_tablet_manager.cc:1939] T 51d4246ebdfa4217a3aea457b2fb1c71 P 7d6de65e1e71410bb07f0b82f068c088: Deleting consensus metadata
I20260805 23:15:55.855489 31787 catalog_manager.cc:4841] TS 7d6de65e1e71410bb07f0b82f068c088 (127.23.88.67:36601): tablet 51d4246ebdfa4217a3aea457b2fb1c71 (table test-table [id=5174143e52d94f6badbb2b83c43e34ff]) successfully deleted
I20260805 23:15:55.855454 31788 catalog_manager.cc:4841] TS d61ee8e651624026bd77bab9fd78d821 (127.23.88.66:40531): tablet 51d4246ebdfa4217a3aea457b2fb1c71 (table test-table [id=5174143e52d94f6badbb2b83c43e34ff]) successfully deleted
W20260805 23:15:55.861548 31787 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:37881 (error 108)
I20260805 23:15:55.861745 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 32027
I20260805 23:15:55.864638 32435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:55.863635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37881 (local address 127.0.0.1:48784)
0805 23:15:55.864398 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:55.864492 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":73,"threads_started":1}
W20260805 23:15:55.865067 31787 catalog_manager.cc:4584] TS cc45e232208949ccb94739a0f9974e6d (127.23.88.65:37881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 51d4246ebdfa4217a3aea457b2fb1c71: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37881: connect: Connection refused (error 111)
I20260805 23:15:55.871950 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 32176
I20260805 23:15:55.879861 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 31769
2026-08-05T23:15:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3230 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[32438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[32438](info): set up 2 sockets
Aug 05 23:15:55 dist-test-slave-vkjp krb5kdc[32438](info): commencing operation
krb5kdc: starting...
W20260805 23:15:56.953218 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:57 dist-test-slave-vkjp krb5kdc[32438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:57.082772 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42071
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:57.197497 32452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:57.197840 32452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:57.197911 32452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:57.202297 32452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:57.202409 32452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:57.202450 32452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:57.202507 32452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:57.202536 32452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:57.208235 32452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:57.209440 32452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:57.210536 32452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:57.219282 32460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:57.219357 32452 server_base.cc:1034] running on GCE node
W20260805 23:15:57.219309 32458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:57.219285 32457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:57.220263 32452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:57.221390 32452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:57.222594 32452 hybrid_clock.cc:648] HybridClock initialized: now 1785971757222576 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-05T23:15:57Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1348 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:15:57 dist-test-slave-vkjp krb5kdc[32466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:15:57 dist-test-slave-vkjp krb5kdc[32466](info): set up 2 sockets
Aug 05 23:15:57 dist-test-slave-vkjp krb5kdc[32466](info): commencing operation
krb5kdc: starting...
W20260805 23:15:58.309778 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:15:58 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:15:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:15:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:15:58.444813 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42183
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:58.563565 32480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:58.563886 32480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:58.563938 32480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:58.568572 32480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:58.568672 32480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:58.568693 32480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:58.568712 32480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:58.568728 32480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:58.574541 32480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:58.575753 32480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:58.576812 32480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:58.586316 32488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:58.586535 32480 server_base.cc:1034] running on GCE node
W20260805 23:15:58.586304 32486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:58.586304 32485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:58.587584 32480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:58.589051 32480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:58.590297 32480 hybrid_clock.cc:648] HybridClock initialized: now 1785971758590283 us; error 47 us; skew 500 ppm
Aug 05 23:15:58 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971758, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:58.593534 32480 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:58.594528 32480 webserver.cc:460] Webserver started at http://127.23.88.126:46689/ using document root <none> and password file <none>
I20260805 23:15:58.595155 32480 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:58.595227 32480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:58.595415 32480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:58.597579 32480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "5a70ae820bd74eea8fec1cd6fffec49a"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "46309a28e89bd049c7f71e23f4c314c5"
server_key_iv: "e39e3d1abe8b40005e515836aabb903b"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.598155 32480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "5a70ae820bd74eea8fec1cd6fffec49a"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "46309a28e89bd049c7f71e23f4c314c5"
server_key_iv: "e39e3d1abe8b40005e515836aabb903b"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.602281 32480 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260805 23:15:58.605118 32495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:58.606509 32480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:15:58.606673 32480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "5a70ae820bd74eea8fec1cd6fffec49a"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "46309a28e89bd049c7f71e23f4c314c5"
server_key_iv: "e39e3d1abe8b40005e515836aabb903b"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.606806 32480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:58.619961 32480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:58.623536 32480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:58.623765 32480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:58.631407 32480 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42183
I20260805 23:15:58.631438 32552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42183 every 8 connection(s)
I20260805 23:15:58.632439 32480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:58.636030 32553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:15:58.642454 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 32480
I20260805 23:15:58.642628 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:15:58.642953 23905 external_mini_cluster.cc:1428] Setting key 6c1ab002c2b1fa63eddd3409dee93eef
I20260805 23:15:58.643157 32553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Bootstrap starting.
I20260805 23:15:58.645965 32553 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Neither blocks nor log segments found. Creating new log.
I20260805 23:15:58.646791 32553 log.cc:824] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:58.648880 32553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: No bootstrap required, opened a new log
Aug 05 23:15:58 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971758, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:58.652944 32553 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:58.653254 32553 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:15:58.653365 32553 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a70ae820bd74eea8fec1cd6fffec49a, State: Initialized, Role: FOLLOWER
I20260805 23:15:58.653949 32553 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:58.654137 32553 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:58.654240 32553 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:58.654348 32553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:15:58.655543 32553 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:58.655915 32553 leader_election.cc:302] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a70ae820bd74eea8fec1cd6fffec49a; no voters:
I20260805 23:15:58.656278 32553 leader_election.cc:288] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:15:58.656502 32559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:15:58.656869 32559 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 LEADER]: Becoming Leader. State: Replica: 5a70ae820bd74eea8fec1cd6fffec49a, State: Running, Role: LEADER
I20260805 23:15:58.657317 32559 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:58.657694 32553 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:58.659296 32560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } } }
I20260805 23:15:58.659456 32560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:58.660100 32561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a70ae820bd74eea8fec1cd6fffec49a. Latest consensus state: current_term: 1 leader_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } } }
I20260805 23:15:58.660218 32561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:58.660528 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:58.644522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.23.88.126:42183)
0805 23:15:58.645038 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:58.645056 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:58.645097 (+ 41us) server_negotiation.cc:407] Connection header received
0805 23:15:58.645881 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:58.645902 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:58.646349 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:58.646730 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:58.647725 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.648840 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.649834 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.650370 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.650565 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:58.653412 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:58.653419 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:58.653447 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:58.653490 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:58.655761 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:58.656441 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:58.656450 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:58.656457 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:58.656638 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:58.657059 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:58.657063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:58.657065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:58.657668 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:58.657912 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:58.658248 (+ 336us) server_negotiation.cc:299] Negotiation successful
0805 23:15:58.658806 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":152,"threads_started":1}
W20260805 23:15:58.663712 32575 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:15:58.663811 32575 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:15:58.663892 32576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:58.664711 32576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:58.670996 32576 catalog_manager.cc:1347] Generated new cluster ID: df491bc5ad084376bdecf9dd18fef97c
I20260805 23:15:58.671126 32576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:58.684347 32576 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:15:58.685734 32576 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:58.693069 32576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Generated new TSK 0
I20260805 23:15:58.693879 32576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:15:58.732831 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:58.854807 32581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:58.855219 32581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:58.855306 32581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:58.859921 32581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:58.859995 32581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:58.860060 32581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:58.865844 32581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:58.867127 32581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:58.868275 32581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:58.878815 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:58.878809 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:58.878842 32587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:58.879112 32581 server_base.cc:1034] running on GCE node
I20260805 23:15:58.879658 32581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:58.880288 32581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:58.881502 32581 hybrid_clock.cc:648] HybridClock initialized: now 1785971758881492 us; error 36 us; skew 500 ppm
Aug 05 23:15:58 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971758, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:58.884573 32581 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:15:58.885568 32581 webserver.cc:460] Webserver started at http://127.23.88.65:46547/ using document root <none> and password file <none>
I20260805 23:15:58.886137 32581 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:58.886199 32581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:58.886376 32581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:58.888545 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "f85166047f454fdfbe90502878575279"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "b620526b8831145233ac6d2bad4dfe4c"
server_key_iv: "9fa9f158f4617ce58195cd6b7e5316a7"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.889107 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "f85166047f454fdfbe90502878575279"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "b620526b8831145233ac6d2bad4dfe4c"
server_key_iv: "9fa9f158f4617ce58195cd6b7e5316a7"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.893195 32581 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:15:58.895969 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:58.897368 32581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:58.897532 32581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "f85166047f454fdfbe90502878575279"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "b620526b8831145233ac6d2bad4dfe4c"
server_key_iv: "9fa9f158f4617ce58195cd6b7e5316a7"
server_key_version: "encryptionkey@0"
I20260805 23:15:58.897639 32581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:58.916834 32581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:58.919970 32581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:58.920188 32581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:58.920810 32581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:58.921645 32581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:58.921692 32581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:58.921734 32581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:58.921749 32581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:58.931638 32581 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:46847
I20260805 23:15:58.931664 32717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:46847 every 8 connection(s)
I20260805 23:15:58.932619 32581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:58.940781 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 32581
I20260805 23:15:58.940964 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:15:58.941313 23905 external_mini_cluster.cc:1428] Setting key 9c0a7841a21b3e78198647018767d466
Aug 05 23:15:58 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971758, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:58.947955 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:58.934522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:51071 (local address 127.23.88.126:42183)
0805 23:15:58.934690 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:58.934694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:58.935687 (+ 993us) server_negotiation.cc:407] Connection header received
0805 23:15:58.937027 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:58.937030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:58.937103 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:58.937180 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:58.938441 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.938995 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.940281 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.940561 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.940685 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:58.943979 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:58.943983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:58.943985 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:58.944022 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:58.945588 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:58.946169 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:58.946173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:58.946174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:58.946262 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:58.946648 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:58.946653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:58.946654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:58.946860 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:58.946983 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:58.947618 (+ 635us) server_negotiation.cc:299] Negotiation successful
0805 23:15:58.947765 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:15:58.948868 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:58.934905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:51071)
0805 23:15:58.935500 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:58.935548 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:58.936794 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:58.937363 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:58.937370 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:58.937767 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:58.938270 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:58.938283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.939169 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.939179 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:58.940140 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:58.940150 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:58.940745 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:58.940749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:58.940956 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:58.941734 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:58.941776 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:58.943780 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:58.945730 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:58.945737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:58.945756 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:58.946054 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:58.946371 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:58.946374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:58.946376 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:58.946546 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:58.946982 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:58.946989 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:58.947354 (+ 365us) client_negotiation.cc:769] Sending connection context
0805 23:15:58.947577 (+ 223us) client_negotiation.cc:240] Negotiation successful
0805 23:15:58.947956 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":197,"threads_started":1}
I20260805 23:15:58.950235 32718 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42183
I20260805 23:15:58.950536 32718 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:58.951287 32718 heartbeater.cc:511] Master 127.23.88.126:42183 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:15:58.953037 32510 ts_manager.cc:194] Registered new tserver with Master: f85166047f454fdfbe90502878575279 (127.23.88.65:46847)
I20260805 23:15:58.954317 32510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:51071
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:15:58.974619 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:59.093122 32723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:59.093425 32723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:59.093472 32723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:59.097661 32723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:59.097729 32723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:59.097792 32723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:15:59.103857 32723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:59.105125 32723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:59.106295 32723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:59.118084 32729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.118084 32731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.118126 32728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:59.118469 32723 server_base.cc:1034] running on GCE node
I20260805 23:15:59.118979 32723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:59.119738 32723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:59.120975 32723 hybrid_clock.cc:648] HybridClock initialized: now 1785971759120931 us; error 53 us; skew 500 ppm
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:59.128738 32723 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:15:59.129850 32723 webserver.cc:460] Webserver started at http://127.23.88.66:41587/ using document root <none> and password file <none>
I20260805 23:15:59.130455 32723 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:59.130527 32723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:59.130712 32723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:59.133036 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "732de9a3b9c3444989fccab3bbe2e729"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "f8761ae66a0cce019bee4be06f1a6cd8"
server_key_iv: "06d11225f358d70a4b8cda71e7b477b4"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.133636 32723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "732de9a3b9c3444989fccab3bbe2e729"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "f8761ae66a0cce019bee4be06f1a6cd8"
server_key_iv: "06d11225f358d70a4b8cda71e7b477b4"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.137859 32723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:15:59.140718 32739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.142153 32723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:15:59.142259 32723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "732de9a3b9c3444989fccab3bbe2e729"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "f8761ae66a0cce019bee4be06f1a6cd8"
server_key_iv: "06d11225f358d70a4b8cda71e7b477b4"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.142359 32723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:59.156803 32723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:59.160504 32723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:59.160705 32723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:59.161363 32723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:59.162351 32723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:59.162406 32723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.162478 32723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:59.162518 32723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.171940 32723 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:36965
I20260805 23:15:59.171969 391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:36965 every 8 connection(s)
I20260805 23:15:59.172890 32723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:15:59.182407 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 32723
I20260805 23:15:59.182572 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:59.182916 23905 external_mini_cluster.cc:1428] Setting key d25c30cc4026e42bb1c461ca453046f2
I20260805 23:15:59.188004 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.174807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:52631 (local address 127.23.88.126:42183)
0805 23:15:59.174998 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:59.175003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:59.175930 (+ 927us) server_negotiation.cc:407] Connection header received
0805 23:15:59.177290 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:59.177293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:59.177343 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:59.177422 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:59.178681 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.179315 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.180458 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.180674 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.180761 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.183600 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:59.183604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:59.183607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:59.183637 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:59.185434 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:59.186009 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:59.186016 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:59.186021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:59.186116 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:59.186681 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:59.186684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:59.186685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:59.186866 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:59.186952 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:59.187750 (+ 798us) server_negotiation.cc:299] Negotiation successful
0805 23:15:59.187863 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260805 23:15:59.189267 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.175293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:52631)
0805 23:15:59.175770 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:59.175816 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:59.177076 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:59.177592 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:59.177599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:59.177985 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:59.178522 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.178535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.179436 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.179439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.180328 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.180337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.180824 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.180827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.180999 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.181606 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:59.181622 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:59.183390 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:59.185566 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:59.185572 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:59.185586 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:59.185878 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:59.186329 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:59.186334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:59.186337 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:59.186575 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:59.186975 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:59.186983 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:59.187449 (+ 466us) client_negotiation.cc:769] Sending connection context
0805 23:15:59.187696 (+ 247us) client_negotiation.cc:240] Negotiation successful
0805 23:15:59.188080 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":134,"threads_started":1}
I20260805 23:15:59.190570 392 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42183
I20260805 23:15:59.190809 392 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:59.191412 392 heartbeater.cc:511] Master 127.23.88.126:42183 requested a full tablet report, sending...
I20260805 23:15:59.192500 32510 ts_manager.cc:194] Registered new tserver with Master: 732de9a3b9c3444989fccab3bbe2e729 (127.23.88.66:36965)
I20260805 23:15:59.193168 32510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:52631
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:15:59.215924 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:59.328218 398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:59.328531 398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:59.328579 398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:59.332685 398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:59.332755 398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:59.332818 398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:15:59.338313 398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:59.339529 398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:59.340556 398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:59.350472 403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.350466 406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.350466 404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:59.350988 398 server_base.cc:1034] running on GCE node
I20260805 23:15:59.351496 398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:59.352269 398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:59.353442 398 hybrid_clock.cc:648] HybridClock initialized: now 1785971759353420 us; error 32 us; skew 500 ppm
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:59.356608 398 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:15:59.357722 398 webserver.cc:460] Webserver started at http://127.23.88.67:46085/ using document root <none> and password file <none>
I20260805 23:15:59.358433 398 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:59.358541 398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:59.358814 398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:15:59.361387 398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "4fedc35662214bb7ad57b519d5bd72dd"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "8eedc9542fe8fe80e9bc78dec311c04f"
server_key_iv: "043b23b0dc28eaea03ae5e81556f70c6"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.362047 398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "4fedc35662214bb7ad57b519d5bd72dd"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "8eedc9542fe8fe80e9bc78dec311c04f"
server_key_iv: "043b23b0dc28eaea03ae5e81556f70c6"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.366452 398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260805 23:15:59.369383 413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.370709 398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:59.370893 398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "4fedc35662214bb7ad57b519d5bd72dd"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "8eedc9542fe8fe80e9bc78dec311c04f"
server_key_iv: "043b23b0dc28eaea03ae5e81556f70c6"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.371032 398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:59.387187 398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:59.390370 398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:59.390602 398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:59.391315 398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:59.392175 398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:59.392222 398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.392264 398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:59.392279 398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.401374 398 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:46075
I20260805 23:15:59.401392 533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:46075 every 8 connection(s)
I20260805 23:15:59.402413 398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:15:59.402905 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 398
I20260805 23:15:59.403047 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:15:59.403396 23905 external_mini_cluster.cc:1428] Setting key a4c7e37e05c2d4aac39652f4e93bea65
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:15:59.416792 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.404315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:60489 (local address 127.23.88.126:42183)
0805 23:15:59.404469 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:59.404474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:59.405680 (+ 1206us) server_negotiation.cc:407] Connection header received
0805 23:15:59.407046 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:59.407049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:59.407165 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:59.407267 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:59.408442 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.408998 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.410135 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.410397 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.410544 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.413216 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:15:59.413220 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:15:59.413222 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:15:59.413247 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:15:59.414618 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:59.415145 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:59.415151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:59.415153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:59.415238 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:15:59.415580 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:15:59.415583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:15:59.415584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:15:59.415796 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:15:59.415908 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0805 23:15:59.416498 (+ 590us) server_negotiation.cc:299] Negotiation successful
0805 23:15:59.416627 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:15:59.417846 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.404888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:60489)
0805 23:15:59.405517 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:59.405581 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:59.406826 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:59.407429 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:59.407435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:59.407808 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:59.408278 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.408291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.409121 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.409123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.410014 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.410022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.410595 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.410598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.410786 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.411454 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:59.411471 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:59.413031 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:15:59.414727 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:59.414732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:59.414745 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:59.414999 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:59.415325 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:15:59.415327 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:15:59.415329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:15:59.415471 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:15:59.415916 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:15:59.415928 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:15:59.416251 (+ 323us) client_negotiation.cc:769] Sending connection context
0805 23:15:59.416473 (+ 222us) client_negotiation.cc:240] Negotiation successful
0805 23:15:59.416806 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":125,"threads_started":1}
I20260805 23:15:59.418991 534 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42183
I20260805 23:15:59.419268 534 heartbeater.cc:464] Registering TS with master...
I20260805 23:15:59.419796 534 heartbeater.cc:511] Master 127.23.88.126:42183 requested a full tablet report, sending...
I20260805 23:15:59.420847 32510 ts_manager.cc:194] Registered new tserver with Master: 4fedc35662214bb7ad57b519d5bd72dd (127.23.88.67:46075)
I20260805 23:15:59.421412 32510 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:60489
I20260805 23:15:59.430263 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
Entry for principal oryx/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab.
WARNING: no policy specified for oryx/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
Entry for principal oryx/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab.
WARNING: no policy specified for oryx/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
Entry for principal oryx/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab.
I20260805 23:15:59.513898 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 32581
I20260805 23:15:59.520731 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 32723
I20260805 23:15:59.527004 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 398
I20260805 23:15:59.534576 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 32480
I20260805 23:15:59.542090 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42183
--webserver_interface=127.23.88.126
--webserver_port=46689
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42183
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:15:59.663369 543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:59.663712 543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:59.663798 543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:59.668126 543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:15:59.668221 543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:59.668255 543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:59.668308 543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:15:59.668346 543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:15:59.674039 543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=46689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:59.675319 543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:59.676384 543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:59.685164 548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.685173 549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:59.685499 543 server_base.cc:1034] running on GCE node
W20260805 23:15:59.685177 552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:59.686275 543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:59.687548 543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:59.688750 543 hybrid_clock.cc:648] HybridClock initialized: now 1785971759688714 us; error 63 us; skew 500 ppm
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:59.691928 543 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:15:59.692906 543 webserver.cc:460] Webserver started at http://127.23.88.126:46689/ using document root <none> and password file <none>
I20260805 23:15:59.693522 543 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:59.693640 543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:59.697777 543 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:15:59.700326 559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.701704 543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:15:59.701867 543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "5a70ae820bd74eea8fec1cd6fffec49a"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "46309a28e89bd049c7f71e23f4c314c5"
server_key_iv: "e39e3d1abe8b40005e515836aabb903b"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.702481 543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:59.714881 543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:59.718035 543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:59.718255 543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:59.725485 543 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42183
I20260805 23:15:59.725556 615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42183 every 8 connection(s)
I20260805 23:15:59.726616 543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:15:59.729015 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 543
I20260805 23:15:59.729491 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:46847
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:42183
--webserver_port=46547
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:15:59.731338 616 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:15:59.733120 616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Bootstrap starting.
I20260805 23:15:59.751000 616 log.cc:824] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:15:59.757835 616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:15:59.758279 616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Bootstrap complete.
I20260805 23:15:59.762665 616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:59.763128 616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a70ae820bd74eea8fec1cd6fffec49a, State: Initialized, Role: FOLLOWER
I20260805 23:15:59.763676 616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:59.763842 616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:15:59.763939 616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:15:59.764061 616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:15:59.766088 616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:59.766438 616 leader_election.cc:302] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a70ae820bd74eea8fec1cd6fffec49a; no voters:
I20260805 23:15:59.766790 616 leader_election.cc:288] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:15:59.766901 620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:15:59.767117 620 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [term 2 LEADER]: Becoming Leader. State: Replica: 5a70ae820bd74eea8fec1cd6fffec49a, State: Running, Role: LEADER
I20260805 23:15:59.767432 620 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } }
I20260805 23:15:59.768113 616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:15:59.768833 621 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } } }
I20260805 23:15:59.768961 621 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:59.769356 622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a70ae820bd74eea8fec1cd6fffec49a. Latest consensus state: current_term: 2 leader_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a70ae820bd74eea8fec1cd6fffec49a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42183 } } }
I20260805 23:15:59.769441 622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a [sys.catalog]: This master's current role is: LEADER
I20260805 23:15:59.770124 627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:15:59.772965 627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:15:59.773562 627 catalog_manager.cc:1259] Loaded cluster ID: df491bc5ad084376bdecf9dd18fef97c
I20260805 23:15:59.773643 627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:15:59.775501 627 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:15:59.776077 627 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5a70ae820bd74eea8fec1cd6fffec49a: Loaded TSK: 0
I20260805 23:15:59.777046 627 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:15:59.855916 618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:15:59.856232 618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:15:59.856328 618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:15:59.860579 618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:15:59.860683 618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:15:59.860781 618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:15:59.866566 618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.65.keytab
--principal=oryx/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:46847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=46547
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:15:59.867878 618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:15:59.869338 618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:15:59.879880 642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.879942 644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:15:59.880313 646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:15:59.880394 618 server_base.cc:1034] running on GCE node
I20260805 23:15:59.880883 618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:15:59.881757 618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:15:59.882977 618 hybrid_clock.cc:648] HybridClock initialized: now 1785971759882949 us; error 39 us; skew 500 ppm
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971759, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:15:59.886786 618 init.cc:377] Logged in from keytab as oryx/127.23.88.65@KRBTEST.COM (short username oryx)
I20260805 23:15:59.888025 618 webserver.cc:460] Webserver started at http://127.23.88.65:46547/ using document root <none> and password file <none>
I20260805 23:15:59.888772 618 fs_manager.cc:359] Metadata directory not provided
I20260805 23:15:59.888906 618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:15:59.894305 618 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260805 23:15:59.897049 653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.898643 618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:15:59.898784 618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "f85166047f454fdfbe90502878575279"
format_stamp: "Formatted at 2026-08-05 23:15:58 on dist-test-slave-vkjp"
server_key: "b620526b8831145233ac6d2bad4dfe4c"
server_key_iv: "9fa9f158f4617ce58195cd6b7e5316a7"
server_key_version: "encryptionkey@0"
I20260805 23:15:59.899598 618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:15:59.924530 618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:15:59.928503 618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:15:59.928721 618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:15:59.929445 618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:15:59.930433 618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:15:59.930492 618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.930540 618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:15:59.930560 618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:15:59.940917 618 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:46847
I20260805 23:15:59.940939 771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:46847 every 8 connection(s)
I20260805 23:15:59.942059 618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:15:59.947597 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 618
I20260805 23:15:59.948148 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:36965
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:42183
--webserver_port=41587
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:15:59.957739 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:59.944322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:35875 (local address 127.23.88.126:42183)
0805 23:15:59.944974 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:59.944992 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:59.945037 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:15:59.946624 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:59.946635 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:59.946968 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:59.947485 (+ 517us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:59.948834 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.950231 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.951646 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.952262 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.952472 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.956952 (+ 4480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:35875: BlockingRecv error: recv got EOF from 127.23.88.65:35875 (error 108)
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":154,"threads_started":1}
I20260805 23:15:59.957835 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.944424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:35875)
0805 23:15:59.944911 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:59.944957 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:59.946177 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:59.947569 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:59.947575 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:59.948039 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:59.948622 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.948639 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.950443 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.950447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.951486 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.951500 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.952475 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.952479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.952698 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.953541 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:59.953558 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:59.956671 (+ 3113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":133,"threads_started":1}
W20260805 23:15:59.958626 772 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:42183 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.88.126:42183: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:15:59.965240 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.959190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:44799)
0805 23:15:59.959362 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:59.959379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:59.959489 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:59.959772 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:59.959775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:59.959971 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:59.960100 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.960107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.960857 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.960860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.961786 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.961797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.962330 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.962334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.962437 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.962967 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:59.962981 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:59.965130 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":65}
W20260805 23:15:59.965456 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:59.959306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:44799 (local address 127.23.88.126:42183)
0805 23:15:59.959413 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:59.959417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:59.959439 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:15:59.959604 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:59.959606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:59.959653 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:59.959724 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:59.960214 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.960750 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.961922 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.962203 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.962320 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.965327 (+ 3007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:44799: BlockingRecv error: recv got EOF from 127.23.88.65:44799 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:15:59 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:15:59.972465 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:15:59.966199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:33351)
0805 23:15:59.966409 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 23:15:59.966421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:15:59.966488 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:15:59.966737 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:15:59.966739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:15:59.966869 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:15:59.966961 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.966966 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.968017 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.968020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.968536 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:15:59.968544 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.969136 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.969138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:15:59.969198 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.970423 (+ 1225us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:15:59.970435 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:15:59.972278 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129}
W20260805 23:15:59.972577 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:15:59.966315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33351 (local address 127.23.88.126:42183)
0805 23:15:59.966444 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:15:59.966448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:15:59.966464 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:15:59.966576 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:15:59.966581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:15:59.966631 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:15:59.966713 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:15:59.967063 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.967893 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.968666 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:15:59.969023 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:15:59.969124 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:15:59.972456 (+ 3332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:33351: BlockingRecv error: recv got EOF from 127.23.88.65:33351 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260805 23:15:59.972733 772 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:16:00.066061 776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:00.066418 776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:00.066510 776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:00.071020 776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:00.071144 776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:00.071247 776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:16:00.077129 776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.66.keytab
--principal=oryx/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:36965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=41587
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:00.078265 776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:00.079377 776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:00.089095 781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:00.089124 784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:00.089094 782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:00.089375 776 server_base.cc:1034] running on GCE node
I20260805 23:16:00.089866 776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:00.090531 776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:00.091720 776 hybrid_clock.cc:648] HybridClock initialized: now 1785971760091704 us; error 36 us; skew 500 ppm
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971760, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:00.094580 776 init.cc:377] Logged in from keytab as oryx/127.23.88.66@KRBTEST.COM (short username oryx)
I20260805 23:16:00.095532 776 webserver.cc:460] Webserver started at http://127.23.88.66:41587/ using document root <none> and password file <none>
I20260805 23:16:00.096072 776 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:00.096150 776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:00.100018 776 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:16:00.102114 791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.103461 776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:16:00.103557 776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "732de9a3b9c3444989fccab3bbe2e729"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "f8761ae66a0cce019bee4be06f1a6cd8"
server_key_iv: "06d11225f358d70a4b8cda71e7b477b4"
server_key_version: "encryptionkey@0"
I20260805 23:16:00.104187 776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:00.122833 776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:00.125936 776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:00.126164 776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:00.126791 776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:00.127699 776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:00.127771 776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.127837 776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:00.127866 776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.137341 776 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:36965
I20260805 23:16:00.137368 912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:36965 every 8 connection(s)
I20260805 23:16:00.138386 776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:16:00.145634 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 776
I20260805 23:16:00.146096 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:46075
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:42183
--webserver_port=46085
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:35083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:16:00.150309 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.140230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:44411 (local address 127.23.88.126:42183)
0805 23:16:00.140372 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.140376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.141216 (+ 840us) server_negotiation.cc:407] Connection header received
0805 23:16:00.142596 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.142600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.142666 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.142771 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.144024 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.144580 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.145761 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.146019 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.146129 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.150194 (+ 4065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:44411: BlockingRecv error: recv got EOF from 127.23.88.66:44411 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:16:00.151271 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.140614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:44411)
0805 23:16:00.141068 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.141112 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.142367 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.142950 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.142957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.143385 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.143871 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.143884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.144723 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.144726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.145616 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.145626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.146190 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.146194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.146396 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.147044 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.147061 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.150045 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":120,"threads_started":1}
W20260805 23:16:00.151813 913 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:42183 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.88.126:42183: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:00.157874 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.152283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:55297)
0805 23:16:00.152496 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.152513 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.152607 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.152896 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.152899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.153081 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.153182 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.153188 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.153940 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.153945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.154492 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.154501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.155016 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.155020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.155145 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.155704 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.155715 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.157672 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133}
W20260805 23:16:00.157912 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.152431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:55297 (local address 127.23.88.126:42183)
0805 23:16:00.152584 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.152588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.152603 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:00.152741 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.152746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.152795 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.152879 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.153303 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.153826 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.154669 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.154894 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.155009 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.157810 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:55297: BlockingRecv error: recv got EOF from 127.23.88.66:55297 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:00.164283 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.158585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:51933)
0805 23:16:00.158802 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.158814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.158883 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.159180 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.159183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.159323 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.159409 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.159414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.160506 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.160509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.161133 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.161143 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.161729 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.161731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.161792 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.162329 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.162341 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.164171 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131}
W20260805 23:16:00.164535 913 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:16:00.164758 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.158713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51933 (local address 127.23.88.126:42183)
0805 23:16:00.158846 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.158850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.158865 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:00.158963 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.158965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.159013 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.159123 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.159523 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.160363 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.161251 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.161614 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.161714 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.164409 (+ 2695us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:51933: BlockingRecv error: recv got EOF from 127.23.88.66:51933 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260805 23:16:00.263167 916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:00.263463 916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:00.263509 916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:00.267539 916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:00.267607 916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:00.267668 916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:16:00.273546 916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:35083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/krb5kdc/oryx_127.23.88.67.keytab
--principal=oryx/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:46075
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=46085
--tserver_master_addrs=127.23.88.126:42183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:00.274724 916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:00.275848 916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:00.286602 922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:00.286568 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:00.286888 916 server_base.cc:1034] running on GCE node
W20260805 23:16:00.287104 925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:00.287487 916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:00.288162 916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:00.289320 916 hybrid_clock.cc:648] HybridClock initialized: now 1785971760289276 us; error 57 us; skew 500 ppm
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971760, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:00.292258 916 init.cc:377] Logged in from keytab as oryx/127.23.88.67@KRBTEST.COM (short username oryx)
I20260805 23:16:00.293236 916 webserver.cc:460] Webserver started at http://127.23.88.67:46085/ using document root <none> and password file <none>
I20260805 23:16:00.293875 916 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:00.293998 916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:00.297973 916 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260805 23:16:00.300128 933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.301378 916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:16:00.301522 916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "4fedc35662214bb7ad57b519d5bd72dd"
format_stamp: "Formatted at 2026-08-05 23:15:59 on dist-test-slave-vkjp"
server_key: "8eedc9542fe8fe80e9bc78dec311c04f"
server_key_iv: "043b23b0dc28eaea03ae5e81556f70c6"
server_key_version: "encryptionkey@0"
I20260805 23:16:00.302088 916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:00.314229 916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:00.317688 916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:00.317920 916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:00.318564 916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:00.319482 916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:00.319554 916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.319633 916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:00.319679 916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:00.329022 916 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:46075
I20260805 23:16:00.329039 1054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:46075 every 8 connection(s)
I20260805 23:16:00.330124 916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:16:00.334371 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 916
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
W20260805 23:16:00.343338 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.332113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:52219 (local address 127.23.88.126:42183)
0805 23:16:00.332257 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.332261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.333152 (+ 891us) server_negotiation.cc:407] Connection header received
0805 23:16:00.334923 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.334930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.334997 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.335122 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.336678 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.337270 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.338725 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.339012 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.339175 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.343197 (+ 4022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:52219: BlockingRecv error: recv got EOF from 127.23.88.67:52219 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:16:00.344046 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.332448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:52219)
0805 23:16:00.332964 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.333012 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.334656 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.335349 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.335358 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.335886 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.336522 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.336536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.337405 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.337409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.338592 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.338604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.339271 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.339278 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.339500 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.340106 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.340123 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.343025 (+ 2902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":133,"threads_started":1}
W20260805 23:16:00.344631 1055 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:42183 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.88.126:42183: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
I20260805 23:16:00.345665 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.335589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.23.88.126:42183)
0805 23:16:00.335874 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.335879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.335894 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:00.335933 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.335936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.335982 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.336105 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.336865 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.337492 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.338351 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.338776 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.338862 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.340398 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:00.340434 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:00.340457 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:00.340489 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:00.342676 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:00.343183 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:00.343189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:00.343195 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:00.343340 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:00.343811 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:00.343814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:00.343815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:00.344222 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:00.344428 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:00.345240 (+ 812us) server_negotiation.cc:299] Negotiation successful
0805 23:16:00.345487 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:00.350692 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.345155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:52455)
0805 23:16:00.345330 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.345345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.345496 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.345781 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.345785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.345961 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.346080 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.346086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.346866 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.346869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.347535 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.347545 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.348087 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.348091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.348184 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.348751 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.348770 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.350593 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106}
W20260805 23:16:00.350862 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.345223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:52455 (local address 127.23.88.126:42183)
0805 23:16:00.345331 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.345337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.345401 (+ 64us) server_negotiation.cc:407] Connection header received
0805 23:16:00.345599 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.345602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.345649 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.345735 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.346189 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.346756 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.347678 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.347967 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.348094 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.350713 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:52455: BlockingRecv error: recv got EOF from 127.23.88.67:52455 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:00.356072 1057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.351308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:38389)
0805 23:16:00.351419 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.351434 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.351535 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.351789 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.351792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.351945 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.352009 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.352014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.352697 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.352702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.353213 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.353221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.353645 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.353649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.353724 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.354181 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.354191 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.355929 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:16:00.356164 774 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.351434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:38389 (local address 127.23.88.126:42183)
0805 23:16:00.351611 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.351615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.351628 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:16:00.351661 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.351663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.351695 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.351776 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.352116 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.352594 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.353326 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.353538 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.353644 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.356050 (+ 2406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:38389: BlockingRecv error: recv got EOF from 127.23.88.67:38389 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260805 23:16:00.356331 1055 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:00 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:00.980073 1060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:00.973508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:34393)
0805 23:16:00.973917 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:00.973933 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:00.974052 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:00.974479 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:00.974482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:00.974682 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:00.974797 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.974803 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.975691 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.975694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.976286 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:00.976296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.976886 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.976905 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:00.976974 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.977649 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:00.977662 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:00.979922 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":204,"threads_started":1}
W20260805 23:16:00.980485 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:00.973654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:34393 (local address 127.23.88.126:42183)
0805 23:16:00.973946 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:00.973950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:00.974019 (+ 69us) server_negotiation.cc:407] Connection header received
0805 23:16:00.974236 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:00.974240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:00.974293 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:00.974393 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:00.974954 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.975572 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.976421 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:00.976759 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:00.976867 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:00.980147 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:34393: BlockingRecv error: recv got EOF from 127.23.88.65:34393 (error 108)
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":120,"threads_started":1}
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:01.171614 1062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:01.165347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:48083)
0805 23:16:01.165721 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:01.165739 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:01.165911 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:01.166208 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:01.166211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:01.166436 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:01.166536 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.166542 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.167396 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.167402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.168016 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.168025 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.168546 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.168550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.168648 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.169203 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:01.169215 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:01.171473 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":98,"threads_started":1}
W20260805 23:16:01.171913 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:01.165522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:48083 (local address 127.23.88.126:42183)
0805 23:16:01.165719 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:01.165724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:01.165842 (+ 118us) server_negotiation.cc:407] Connection header received
0805 23:16:01.166029 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:01.166032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:01.166090 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:01.166179 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:01.166668 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.167287 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.168148 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.168421 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.168528 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.171636 (+ 3108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:48083: BlockingRecv error: recv got EOF from 127.23.88.66:48083 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:01.363344 1063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:01.357075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:58351)
0805 23:16:01.357432 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:01.357447 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:01.357560 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:01.357809 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:01.357811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:01.358004 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:01.358111 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.358117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.359005 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.359008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.359702 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.359714 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.360282 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.360285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.360354 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.361026 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:01.361053 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:01.363191 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":123,"threads_started":1}
W20260805 23:16:01.363472 1061 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:01.357209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:58351 (local address 127.23.88.126:42183)
0805 23:16:01.357420 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:01.357424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:01.357524 (+ 100us) server_negotiation.cc:407] Connection header received
0805 23:16:01.357633 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:01.357635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:01.357686 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:01.357762 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:01.358257 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.358871 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.359847 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.360120 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.360214 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.363329 (+ 3115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:58351: BlockingRecv error: recv got EOF from 127.23.88.67:58351 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:01 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:01.987708 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:01.981271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:49015)
0805 23:16:01.981625 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:01.981645 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:01.981764 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:01.982109 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:01.982112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:01.982330 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:01.982444 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.982451 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.983264 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.983267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.983938 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:01.983948 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.984490 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.984503 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:01.984576 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.985178 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:01.985196 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:01.987581 (+ 2385us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":91,"threads_started":1}
W20260805 23:16:01.988341 1065 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:01.981384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:49015 (local address 127.23.88.126:42183)
0805 23:16:01.981743 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:01.981747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:01.981764 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:16:01.981881 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:01.981883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:01.981933 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:01.982020 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:01.982574 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.983139 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.984089 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:01.984356 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:01.984449 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:01.988146 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:49015: BlockingRecv error: recv got EOF from 127.23.88.65:49015 (error 108)
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":133,"threads_started":1}
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:02.178422 1066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:02.172622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:51781)
0805 23:16:02.172945 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:02.172965 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:02.173107 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:02.173406 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:02.173410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:02.173628 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:02.173726 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.173732 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.174526 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.174531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.175181 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.175197 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.175687 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.175690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.175776 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.176331 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:02.176343 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:02.178296 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":99,"threads_started":1}
W20260805 23:16:02.178632 1065 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:02.172871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51781 (local address 127.23.88.126:42183)
0805 23:16:02.173053 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:02.173058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:02.173077 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:16:02.173223 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:02.173228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:02.173280 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:02.173379 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:02.173854 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.174405 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.175308 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.175559 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.175668 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.178491 (+ 2823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:51781: BlockingRecv error: recv got EOF from 127.23.88.66:51781 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:02.370395 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:02.364267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:36547)
0805 23:16:02.364662 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:02.364682 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:02.364801 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:02.365125 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:02.365128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:02.365341 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:02.365442 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.365447 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.366255 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.366260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.366964 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.366976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.367553 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.367556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.367640 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.368215 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:02.368227 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:02.370274 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":89,"threads_started":1}
W20260805 23:16:02.370625 1065 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:02.364571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:36547 (local address 127.23.88.126:42183)
0805 23:16:02.364736 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:02.364740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:02.364756 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:16:02.364927 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:02.364931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:02.364998 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:02.365093 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:02.365580 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.366132 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.367205 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.367428 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.367547 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.370499 (+ 2952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:36547: BlockingRecv error: recv got EOF from 127.23.88.67:36547 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:02 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:02.995323 1068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:02.988816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:54553)
0805 23:16:02.989236 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:02.989256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:02.989402 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:02.989793 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:02.989797 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:02.990049 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:02.990186 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.990194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.991263 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.991266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.991837 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:02.991845 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.992287 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.992291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:02.992368 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.992893 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:02.992907 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:02.995167 (+ 2260us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":101,"threads_started":1}
W20260805 23:16:02.995488 1070 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:02.989015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:54553 (local address 127.23.88.126:42183)
0805 23:16:02.989322 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:02.989326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:02.989342 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:16:02.989544 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:02.989548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:02.989614 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:02.989720 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:02.990317 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.991139 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.991942 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:02.992175 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:02.992254 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:02.995317 (+ 3063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:54553: BlockingRecv error: recv got EOF from 127.23.88.65:54553 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":147,"threads_started":1}
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:03.185243 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:03.179346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:49271)
0805 23:16:03.179671 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:03.179687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:03.179799 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:03.180218 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:03.180220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:03.180405 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:03.180528 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.180534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.181348 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.181350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:03.181909 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.181917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.182399 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.182401 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:03.182483 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:03.183019 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:03.183030 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:03.185051 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":137,"threads_started":1}
W20260805 23:16:03.185374 1070 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:03.179517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:49271 (local address 127.23.88.126:42183)
0805 23:16:03.179675 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:03.179679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:03.179951 (+ 272us) server_negotiation.cc:407] Connection header received
0805 23:16:03.180004 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:03.180006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:03.180074 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:03.180189 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:03.180642 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.181205 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.182019 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.182287 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.182381 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:03.185235 (+ 2854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:49271: BlockingRecv error: recv got EOF from 127.23.88.66:49271 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:03.377097 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:03.371415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:53671)
0805 23:16:03.371723 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:03.371737 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:03.371853 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:03.372144 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:03.372146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:03.372335 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:03.372448 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.372454 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.373300 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.373303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:03.373894 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.373903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.374371 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.374375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:03.374445 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:03.374989 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:03.375002 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:03.376969 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":118,"threads_started":1}
W20260805 23:16:03.377301 1070 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:03.371471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:53671 (local address 127.23.88.126:42183)
0805 23:16:03.371618 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:03.371622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:03.371801 (+ 179us) server_negotiation.cc:407] Connection header received
0805 23:16:03.371925 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:03.371927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:03.371976 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:03.372054 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:03.372601 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.373184 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.374012 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.374229 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.374321 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:03.377153 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:53671: BlockingRecv error: recv got EOF from 127.23.88.67:53671 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:03 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:04.003943 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:03.996390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:43083)
0805 23:16:03.996771 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:03.996787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:03.996933 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:03.997267 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:03.997270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:03.997538 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:03.997670 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.997677 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.998787 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.998790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:03.999408 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:03.999419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.000070 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.000078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:04.000185 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.000934 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:04.000947 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:04.003746 (+ 2799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":117,"threads_started":1}
W20260805 23:16:04.004130 1075 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:03.996560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:43083 (local address 127.23.88.126:42183)
0805 23:16:03.996821 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:03.996825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:03.996870 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:16:03.997037 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:03.997039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:03.997098 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:03.997183 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:03.997832 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.998620 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:03.999560 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:03.999883 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.000048 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.003935 (+ 3887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:43083: BlockingRecv error: recv got EOF from 127.23.88.65:43083 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":68,"threads_started":1}
Aug 05 23:16:04 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:04 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:04.196879 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:04.186134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:36943)
0805 23:16:04.186464 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:04.186479 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:04.186585 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:04.186833 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:04.186834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:04.187023 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:04.187161 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:04.187167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.187899 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.187901 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:04.188490 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:04.188501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.188958 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.188960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:04.189026 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.194084 (+ 5058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:04.194102 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:04.196704 (+ 2602us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":76,"threads_started":1}
W20260805 23:16:04.196980 1075 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:04.186300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:36943 (local address 127.23.88.126:42183)
0805 23:16:04.186477 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:04.186481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:04.186552 (+ 71us) server_negotiation.cc:407] Connection header received
0805 23:16:04.186669 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:04.186671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:04.186717 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:04.186790 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:04.187259 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.187789 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.188627 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.188843 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.188924 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.196857 (+ 7933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:36943: BlockingRecv error: recv got EOF from 127.23.88.66:36943 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 05 23:16:04 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:04 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:04.385125 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:04.378323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:48459)
0805 23:16:04.378675 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:04.378691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:04.378805 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:04.379209 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:04.379212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:04.379442 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:04.379549 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:04.379554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.380594 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.380597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:04.381453 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:04.381467 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.382002 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.382005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:04.382072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.382704 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:04.382719 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:04.384992 (+ 2273us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":119,"threads_started":1}
W20260805 23:16:04.385325 1075 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:04.378519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:48459 (local address 127.23.88.126:42183)
0805 23:16:04.378711 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:04.378715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:04.378757 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:16:04.378955 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:04.378959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:04.379014 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:04.379130 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:04.379734 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.380446 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.381601 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:04.381852 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:04.381962 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:04.385175 (+ 3213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:48459: BlockingRecv error: recv got EOF from 127.23.88.67:48459 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:05.011479 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:05.004851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:48437)
0805 23:16:05.005215 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:05.005230 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:05.005333 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:05.005586 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:05.005588 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:05.005840 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:05.005979 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.005986 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.007241 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.007244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.007898 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.007909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.008529 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.008531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.008602 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.009260 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:05.009275 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:05.011335 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
W20260805 23:16:05.011719 1079 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:05.004956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:48437 (local address 127.23.88.126:42183)
0805 23:16:05.005225 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:05.005229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:05.005304 (+ 75us) server_negotiation.cc:407] Connection header received
0805 23:16:05.005418 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:05.005421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:05.005466 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:05.005548 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:05.006197 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.007019 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.008052 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.008377 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.008495 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.011544 (+ 3049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:48437: BlockingRecv error: recv got EOF from 127.23.88.65:48437 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":64,"threads_started":1}
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:05.204640 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:05.198253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:47179)
0805 23:16:05.198613 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:05.198633 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:05.198807 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:05.199248 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:05.199251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:05.199517 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:05.199643 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.199654 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.200490 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.200493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.201136 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.201147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.201634 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.201637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.201703 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.202327 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:05.202341 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:05.204459 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":95,"threads_started":1}
W20260805 23:16:05.204790 1079 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:05.198505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:47179 (local address 127.23.88.126:42183)
0805 23:16:05.198699 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:05.198704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:05.198719 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:05.198944 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:05.198948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:05.199007 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:05.199139 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:05.199788 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.200363 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.201259 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.201520 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.201603 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.204667 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:47179: BlockingRecv error: recv got EOF from 127.23.88.66:47179 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:05 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:05.392704 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:05.386326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:54383)
0805 23:16:05.386681 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:05.386696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:05.386830 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:05.387162 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:05.387165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:05.387392 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:05.387499 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.387505 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.388342 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.388345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.388969 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:05.388979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.389477 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.389480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:05.389545 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.390189 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:05.390203 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:05.392550 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":132,"threads_started":1}
W20260805 23:16:05.392972 1079 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:05.386525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:54383 (local address 127.23.88.126:42183)
0805 23:16:05.386699 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:05.386703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:05.386771 (+ 68us) server_negotiation.cc:407] Connection header received
0805 23:16:05.386934 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:05.386937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:05.386986 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:05.387071 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:05.387643 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.388208 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.389126 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:05.389359 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:05.389455 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:05.392789 (+ 3334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:54383: BlockingRecv error: recv got EOF from 127.23.88.67:54383 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:06.018726 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:06.012514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:39609)
0805 23:16:06.012819 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:06.012837 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:06.012944 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:06.013218 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:06.013220 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:06.013439 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:06.013539 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.013545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.014356 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.014359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.014943 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.014951 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.015630 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.015633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.015701 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.016312 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:06.016324 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:06.018604 (+ 2280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":97,"threads_started":1}
W20260805 23:16:06.018949 1083 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:06.012644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:39609 (local address 127.23.88.126:42183)
0805 23:16:06.012938 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:06.012941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:06.012963 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:16:06.013038 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:06.013040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:06.013084 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:06.013165 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:06.013677 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.014223 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.015071 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.015513 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.015608 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.018756 (+ 3148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:39609: BlockingRecv error: recv got EOF from 127.23.88.65:39609 (error 108)
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":128,"threads_started":1}
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:06.211764 1084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:06.205765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:38973)
0805 23:16:06.206093 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:06.206107 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:06.206211 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:06.206495 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:06.206497 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:06.206693 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:06.206831 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.206861 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.207879 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.207882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.208531 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.208542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.208979 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.208981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.209046 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.209633 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:06.209645 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:06.211581 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":103,"threads_started":1}
W20260805 23:16:06.211841 1083 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:06.205875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:38973 (local address 127.23.88.126:42183)
0805 23:16:06.206070 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:06.206074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:06.206199 (+ 125us) server_negotiation.cc:407] Connection header received
0805 23:16:06.206287 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:06.206290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:06.206341 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:06.206459 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:06.206986 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.207753 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.208650 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.208878 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.208966 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.211724 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:38973: BlockingRecv error: recv got EOF from 127.23.88.66:38973 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:06 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:06.400180 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:06.393685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:41113)
0805 23:16:06.394055 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:06.394075 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:06.394236 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:06.394567 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:06.394570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:06.394801 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:06.394903 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.394909 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.395767 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.395772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.396356 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:06.396363 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.396857 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.396860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:06.396957 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.397630 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:06.397643 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:06.400011 (+ 2368us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":90,"threads_started":1}
W20260805 23:16:06.400298 1083 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:06.393808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:41113 (local address 127.23.88.126:42183)
0805 23:16:06.393999 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:06.394005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:06.394173 (+ 168us) server_negotiation.cc:407] Connection header received
0805 23:16:06.394347 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:06.394350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:06.394430 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:06.394524 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:06.395028 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.395651 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.396483 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:06.396736 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:06.396850 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:06.400178 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:41113: BlockingRecv error: recv got EOF from 127.23.88.67:41113 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:07.026265 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:07.019763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:36299)
0805 23:16:07.020250 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:07.020264 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:07.020409 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:07.020813 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:07.020817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:07.021133 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:07.021272 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.021280 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.022150 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.022152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.022722 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.022731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.023212 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.023216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.023341 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.023905 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:07.023917 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:07.026071 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":107,"threads_started":1}
W20260805 23:16:07.026388 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:07.019872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:36299 (local address 127.23.88.126:42183)
0805 23:16:07.020363 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:07.020367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:07.020381 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:16:07.020517 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:07.020519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:07.020605 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:07.020739 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:07.021440 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.022036 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.022832 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.023104 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.023185 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.026231 (+ 3046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:36299: BlockingRecv error: recv got EOF from 127.23.88.65:36299 (error 108)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":118,"threads_started":1}
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:07.218890 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:07.212739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:38861)
0805 23:16:07.213211 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:07.213224 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:07.213367 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:07.213696 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:07.213700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:07.213985 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:07.214102 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.214108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.214893 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.214898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.215648 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.215658 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.216136 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.216140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.216243 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.216829 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:07.216850 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:07.218715 (+ 1865us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":115,"threads_started":1}
W20260805 23:16:07.218959 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:07.212845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:38861 (local address 127.23.88.126:42183)
0805 23:16:07.213035 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:07.213040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:07.213307 (+ 267us) server_negotiation.cc:407] Connection header received
0805 23:16:07.213486 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:07.213491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:07.213556 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:07.213629 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:07.214229 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.214778 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.215778 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.216010 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.216123 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.218847 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:38861: BlockingRecv error: recv got EOF from 127.23.88.66:38861 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:07 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:07.407589 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:07.401058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:60769)
0805 23:16:07.401440 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:07.401454 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:07.401568 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:07.401863 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:07.401865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:07.402059 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:07.402175 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.402180 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.403012 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.403015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.403667 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:07.403677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.404197 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.404201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:07.404271 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.405005 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:07.405018 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:07.407417 (+ 2399us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":80,"threads_started":1}
W20260805 23:16:07.407855 1087 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:07.401241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:60769 (local address 127.23.88.126:42183)
0805 23:16:07.401443 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:07.401446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:07.401561 (+ 115us) server_negotiation.cc:407] Connection header received
0805 23:16:07.401660 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:07.401662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:07.401745 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:07.401814 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:07.402315 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.402896 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.403787 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:07.404039 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:07.404129 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:07.407708 (+ 3579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:60769: BlockingRecv error: recv got EOF from 127.23.88.67:60769 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:08.033456 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:08.027322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:54659)
0805 23:16:08.027702 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:08.027727 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:08.027829 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:08.028123 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:08.028125 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:08.028323 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:08.028459 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.028465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.029275 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.029279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.029942 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.029952 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.030532 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.030536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.030606 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.031226 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:08.031238 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:08.033329 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":132,"threads_started":1}
W20260805 23:16:08.033695 1092 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:08.027539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:54659 (local address 127.23.88.126:42183)
0805 23:16:08.027801 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:08.027804 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:08.027818 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:16:08.027934 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:08.027936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:08.027983 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:08.028064 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:08.028584 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.029142 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.030094 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.030383 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.030496 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.033560 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:54659: BlockingRecv error: recv got EOF from 127.23.88.65:54659 (error 108)
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":83,"threads_started":1}
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:08.226234 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:08.219950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:46461)
0805 23:16:08.220295 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:08.220311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:08.220432 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:08.220709 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:08.220711 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:08.220944 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:08.221082 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.221090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.222085 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.222088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.222771 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.222783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.223370 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.223373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.223446 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.224108 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:08.224121 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:08.226038 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":134,"threads_started":1}
W20260805 23:16:08.226562 1092 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:08.220090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46461 (local address 127.23.88.126:42183)
0805 23:16:08.220248 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:08.220251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:08.220402 (+ 151us) server_negotiation.cc:407] Connection header received
0805 23:16:08.220530 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:08.220533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:08.220580 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:08.220668 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:08.221234 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.221933 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.222925 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.223216 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.223319 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.226257 (+ 2938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:46461: BlockingRecv error: recv got EOF from 127.23.88.66:46461 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:08 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:08.414516 1094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:08.408557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:55711)
0805 23:16:08.408832 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:08.408845 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:08.408999 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:08.409312 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:08.409315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:08.409540 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:08.409642 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.409648 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.410612 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.410617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.411242 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:08.411252 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.411745 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.411749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:08.411837 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.412460 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:08.412472 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:08.414350 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
W20260805 23:16:08.414631 1092 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:08.408599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:55711 (local address 127.23.88.126:42183)
0805 23:16:08.408731 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:08.408735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:08.408930 (+ 195us) server_negotiation.cc:407] Connection header received
0805 23:16:08.409123 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:08.409126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:08.409187 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:08.409279 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:08.409769 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.410474 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.411379 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:08.411614 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:08.411726 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:08.414515 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:55711: BlockingRecv error: recv got EOF from 127.23.88.67:55711 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:09.040417 1095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:09.034500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:58927)
0805 23:16:09.034796 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:09.034809 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:09.034913 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:09.035221 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:09.035223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:09.035421 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:09.035531 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.035537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.036327 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.036330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.036939 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.036950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.037542 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.037547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.037623 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.038181 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:09.038197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:09.040234 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":105,"threads_started":1}
W20260805 23:16:09.040520 1096 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:09.034671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:58927 (local address 127.23.88.126:42183)
0805 23:16:09.034931 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:09.034935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:09.034950 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:09.034996 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:09.034999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:09.035045 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:09.035168 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:09.035658 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.036220 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.037096 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.037413 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.037519 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.040378 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:58927: BlockingRecv error: recv got EOF from 127.23.88.65:58927 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:09.236155 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:09.227288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:41959)
0805 23:16:09.227612 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:09.227627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:09.227741 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:09.228002 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:09.228004 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:09.228219 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:09.228412 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.228422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.229303 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.229308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.229870 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.229878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.230359 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.230362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.230449 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.233988 (+ 3539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:09.234001 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:09.236031 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
W20260805 23:16:09.236346 1096 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:09.227359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:41959 (local address 127.23.88.126:42183)
0805 23:16:09.227553 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:09.227560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:09.227705 (+ 145us) server_negotiation.cc:407] Connection header received
0805 23:16:09.227831 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:09.227834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:09.227882 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:09.227954 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:09.228561 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.229175 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.229998 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.230231 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.230349 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.236184 (+ 5835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:41959: BlockingRecv error: recv got EOF from 127.23.88.66:41959 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:09 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:09.421401 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:09.415523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:44379)
0805 23:16:09.415778 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:09.415795 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:09.415952 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:09.416243 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:09.416246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:09.416462 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:09.416567 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.416572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.417399 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.417404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.417968 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:09.417977 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.418468 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.418472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:09.418567 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.419197 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:09.419209 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:09.421273 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
W20260805 23:16:09.421643 1096 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:09.415641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:44379 (local address 127.23.88.126:42183)
0805 23:16:09.415777 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:09.415782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:09.415888 (+ 106us) server_negotiation.cc:407] Connection header received
0805 23:16:09.416059 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:09.416062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:09.416122 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:09.416215 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:09.416702 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.417279 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.418101 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:09.418346 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:09.418459 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:09.421494 (+ 3035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:44379: BlockingRecv error: recv got EOF from 127.23.88.67:44379 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:10.047246 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:10.041345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:34439)
0805 23:16:10.041663 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:10.041677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:10.041768 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:10.042043 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:10.042046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:10.042239 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:10.042340 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.042346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.043205 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.043208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.043773 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.043782 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.044302 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.044305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.044378 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.044936 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:10.044949 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:10.047001 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":92,"threads_started":1}
W20260805 23:16:10.047377 1100 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:10.041492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:34439 (local address 127.23.88.126:42183)
0805 23:16:10.041724 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:10.041727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:10.041741 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:16:10.041876 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:10.041879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:10.041924 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:10.042018 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:10.042484 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.043046 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.043891 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.044131 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.044213 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.047224 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:34439: BlockingRecv error: recv got EOF from 127.23.88.65:34439 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":61,"threads_started":1}
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:10.242630 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:10.237039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:36891)
0805 23:16:10.237348 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:10.237363 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:10.237469 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:10.237735 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:10.237737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:10.237926 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:10.238026 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.238031 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.238805 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.238808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.239513 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.239523 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.239976 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.239978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.240037 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.240541 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:10.240552 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:10.242484 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":110,"threads_started":1}
W20260805 23:16:10.242761 1100 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:10.237103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:36891 (local address 127.23.88.126:42183)
0805 23:16:10.237228 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:10.237232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:10.237439 (+ 207us) server_negotiation.cc:407] Connection header received
0805 23:16:10.237538 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:10.237540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:10.237585 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:10.237659 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:10.238137 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.238695 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.239635 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.239869 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.239961 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.242635 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:36891: BlockingRecv error: recv got EOF from 127.23.88.66:36891 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:10 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:10.431813 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:10.422515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:57139)
0805 23:16:10.422987 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:10.423005 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:10.423148 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:10.423435 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:10.423438 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:10.423656 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:10.423750 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.423756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.424522 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.424525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.425091 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:10.425099 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.425508 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.425510 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:10.425571 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.429714 (+ 4143us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:10.429727 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:10.431690 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":129,"threads_started":1}
W20260805 23:16:10.431967 1100 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:10.422698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:57139 (local address 127.23.88.126:42183)
0805 23:16:10.422893 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:10.422898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:10.423123 (+ 225us) server_negotiation.cc:407] Connection header received
0805 23:16:10.423255 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:10.423258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:10.423321 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:10.423413 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:10.423866 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.424419 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.425193 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:10.425409 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:10.425487 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:10.431836 (+ 6349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:57139: BlockingRecv error: recv got EOF from 127.23.88.67:57139 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:11.053589 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:11.048144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:39191)
0805 23:16:11.048453 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:11.048466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:11.048568 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:11.048796 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:11.048798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:11.048978 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:11.049078 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.049083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.049821 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.049824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.050364 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.050371 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.050805 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.050807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.050869 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.051404 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:11.051447 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:11.053360 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
W20260805 23:16:11.053637 1105 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:11.048235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:39191 (local address 127.23.88.126:42183)
0805 23:16:11.048474 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:11.048478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:11.048517 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:16:11.048643 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:11.048645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:11.048687 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:11.048757 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:11.049180 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.049726 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.050488 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.050708 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.050783 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.053513 (+ 2730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:39191: BlockingRecv error: recv got EOF from 127.23.88.65:39191 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":85,"threads_started":1}
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:11.249182 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:11.243528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:56641)
0805 23:16:11.243838 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:11.243851 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:11.243987 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:11.244282 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:11.244285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:11.244527 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:11.244627 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.244633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.245402 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.245408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.245971 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.245980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.246460 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.246463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.246548 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.247075 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:11.247140 (+ 65us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:11.249000 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":88,"threads_started":1}
W20260805 23:16:11.249248 1105 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:11.243595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:56641 (local address 127.23.88.126:42183)
0805 23:16:11.243777 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:11.243782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:11.243928 (+ 146us) server_negotiation.cc:407] Connection header received
0805 23:16:11.244100 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:11.244104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:11.244159 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:11.244247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:11.244753 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.245290 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.246096 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.246328 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.246444 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.249134 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:56641: BlockingRecv error: recv got EOF from 127.23.88.66:56641 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:11 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:11.438871 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:11.432957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:44215)
0805 23:16:11.433257 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:11.433274 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:11.433390 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:11.433716 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:11.433719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:11.433961 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:11.434116 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.434124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.435176 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.435179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.435746 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:11.435755 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.436198 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.436200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:11.436260 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.436791 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:11.436808 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:11.438731 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
W20260805 23:16:11.439011 1105 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:11.433032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:44215 (local address 127.23.88.126:42183)
0805 23:16:11.433460 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:11.433463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:11.433478 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:11.433521 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:11.433523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:11.433568 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:11.433648 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:11.434252 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.435029 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.435867 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:11.436097 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:11.436188 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:11.438875 (+ 2687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:44215: BlockingRecv error: recv got EOF from 127.23.88.67:44215 (error 108)
Metrics: {"server-negotiator.queue_time_us":334}
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:12.059844 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:12.054493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:50349)
0805 23:16:12.054829 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:12.054844 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:12.054940 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:12.055211 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:12.055213 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:12.055397 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:12.055494 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.055499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.056236 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.056239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.056793 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.056802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.057251 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.057254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.057318 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.057828 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:12.057841 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:12.059714 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":94,"threads_started":1}
W20260805 23:16:12.060067 1110 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:12.054608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:50349 (local address 127.23.88.126:42183)
0805 23:16:12.054879 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:12.054883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:12.054896 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:16:12.055019 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:12.055022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:12.055097 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:12.055167 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:12.055605 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.056136 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.056917 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.057143 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.057224 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.059907 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:50349: BlockingRecv error: recv got EOF from 127.23.88.65:50349 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":70,"threads_started":1}
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:12.256791 1111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:12.250217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:56603)
0805 23:16:12.250527 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:12.250545 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:12.250670 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:12.250966 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:12.250969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:12.251197 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:12.251346 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.251352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.252366 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.252369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.253227 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.253247 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.253764 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.253767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.253838 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.254561 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:12.254575 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:12.256667 (+ 2092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":116,"threads_started":1}
W20260805 23:16:12.256949 1110 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:12.250292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:56603 (local address 127.23.88.126:42183)
0805 23:16:12.250445 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:12.250449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:12.250630 (+ 181us) server_negotiation.cc:407] Connection header received
0805 23:16:12.250758 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:12.250760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:12.250808 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:12.250902 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:12.251490 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.252230 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.253382 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.253610 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.253699 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.256823 (+ 3124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:56603: BlockingRecv error: recv got EOF from 127.23.88.66:56603 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:12 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:12.445377 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:12.439832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:58173)
0805 23:16:12.440178 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:12.440190 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:12.440316 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:12.440597 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:12.440601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:12.440819 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:12.440914 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.440920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.441708 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.441714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.442268 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:12.442277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.442749 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.442752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:12.442838 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.443393 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:12.443405 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:12.445256 (+ 1851us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":92,"threads_started":1}
W20260805 23:16:12.445592 1110 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:12.439887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:58173 (local address 127.23.88.126:42183)
0805 23:16:12.440035 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:12.440040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:12.440267 (+ 227us) server_negotiation.cc:407] Connection header received
0805 23:16:12.440430 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:12.440434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:12.440489 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:12.440569 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:12.441046 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.441596 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.442396 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:12.442635 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:12.442742 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:12.445452 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:58173: BlockingRecv error: recv got EOF from 127.23.88.67:58173 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:13.066385 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:13.060594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:35435)
0805 23:16:13.060966 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:13.060983 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:13.061110 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:13.061429 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:13.061432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:13.061616 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:13.061718 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.061724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.062529 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.062532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.063126 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.063134 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.063606 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.063609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.063670 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.064231 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:13.064243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:13.066260 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
W20260805 23:16:13.066630 1115 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:13.060648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:35435 (local address 127.23.88.126:42183)
0805 23:16:13.060928 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:13.060934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:13.061067 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:16:13.061214 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:13.061217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:13.061276 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:13.061362 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:13.061833 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.062424 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.063230 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.063505 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.063589 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.066465 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:35435: BlockingRecv error: recv got EOF from 127.23.88.65:35435 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:13.263938 1116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:13.257762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:55433)
0805 23:16:13.258090 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:13.258105 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:13.258210 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:13.258506 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:13.258509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:13.258717 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:13.258828 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.258834 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.259623 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.259626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.260205 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.260213 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.260628 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.260631 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.260693 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.261277 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:13.261297 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:13.263745 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":135,"threads_started":1}
W20260805 23:16:13.264001 1115 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:13.257916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:55433 (local address 127.23.88.126:42183)
0805 23:16:13.258066 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:13.258069 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:13.258176 (+ 107us) server_negotiation.cc:407] Connection header received
0805 23:16:13.258298 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:13.258300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:13.258345 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:13.258422 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:13.258955 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.259502 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.260308 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.260533 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.260607 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.263875 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:55433: BlockingRecv error: recv got EOF from 127.23.88.66:55433 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:13 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:13.452596 1117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:13.446335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:54395)
0805 23:16:13.446666 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:13.446681 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:13.446788 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:13.447174 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:13.447176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:13.447415 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:13.447517 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.447524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.448403 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.448427 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.449016 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:13.449026 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.449545 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.449547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:13.449612 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.450143 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:13.450156 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:13.452407 (+ 2251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":136,"threads_started":1}
W20260805 23:16:13.452711 1115 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:13.446483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:54395 (local address 127.23.88.126:42183)
0805 23:16:13.446656 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:13.446666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:13.446816 (+ 150us) server_negotiation.cc:407] Connection header received
0805 23:16:13.446873 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:13.446875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:13.446933 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:13.447018 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:13.447676 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.448267 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.449160 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:13.449428 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:13.449523 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:13.452587 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:54395: BlockingRecv error: recv got EOF from 127.23.88.67:54395 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:14.073127 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:14.067283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:54605)
0805 23:16:14.067612 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:14.067629 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:14.067738 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:14.067954 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:14.067957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:14.068136 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:14.068240 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.068270 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.069125 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.069129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.069790 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.069799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.070251 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.070254 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.070315 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.070792 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:14.070803 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:14.072994 (+ 2191us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":133,"threads_started":1}
W20260805 23:16:14.073348 1119 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:14.067393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:54605 (local address 127.23.88.126:42183)
0805 23:16:14.067727 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:14.067731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:14.067744 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:16:14.067806 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:14.067808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:14.067850 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:14.067941 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:14.068380 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.068998 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.069906 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.070133 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.070217 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.073159 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:54605: BlockingRecv error: recv got EOF from 127.23.88.65:54605 (error 108)
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":101,"threads_started":1}
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:14.276696 1120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:14.264781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:39315)
0805 23:16:14.265253 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:14.265265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:14.265432 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:14.265715 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:14.265717 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:14.265937 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:14.266034 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.266040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.266805 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.266810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.267433 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.267444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.267993 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.267996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.268106 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.274594 (+ 6488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:14.274609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:14.276559 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":191,"threads_started":1}
W20260805 23:16:14.276912 1119 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:14.264874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:39315 (local address 127.23.88.126:42183)
0805 23:16:14.265127 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:14.265134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:14.265344 (+ 210us) server_negotiation.cc:407] Connection header received
0805 23:16:14.265540 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:14.265543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:14.265597 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:14.265686 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:14.266159 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.266693 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.267564 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.267870 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.267978 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.276755 (+ 8777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:39315: BlockingRecv error: recv got EOF from 127.23.88.66:39315 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:14 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.67@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:14.458877 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:14.453363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.67:47753)
0805 23:16:14.453666 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:14.453678 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:14.453814 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:14.454103 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:14.454106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:14.454312 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:14.454410 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.454415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.455195 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.455200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.455771 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:14.455779 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.456250 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.456253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:14.456338 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.456900 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:14.456912 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:14.458682 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":118,"threads_started":1}
W20260805 23:16:14.458942 1119 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:14.453416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:47753 (local address 127.23.88.126:42183)
0805 23:16:14.453574 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:14.453579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:14.453753 (+ 174us) server_negotiation.cc:407] Connection header received
0805 23:16:14.453928 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:14.453931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:14.453985 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:14.454079 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:14.454533 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.455062 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.455901 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:14.456139 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:14.456244 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:14.458824 (+ 2580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.67:47753: BlockingRecv error: recv got EOF from 127.23.88.67:47753 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 05 23:16:15 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:15 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.65@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:15.079487 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:15.074094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.65:35685)
0805 23:16:15.074409 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:15.074423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:15.074529 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:15.074764 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:15.074766 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:15.074956 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:15.075058 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.075063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.075888 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.075891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.076455 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.076463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.076893 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.076896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.076958 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.077522 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:15.077534 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:15.079366 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":99,"threads_started":1}
W20260805 23:16:15.079778 1123 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:15.074166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:35685 (local address 127.23.88.126:42183)
0805 23:16:15.074425 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:15.074428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:15.074493 (+ 65us) server_negotiation.cc:407] Connection header received
0805 23:16:15.074606 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:15.074608 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:15.074654 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:15.074732 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:15.075214 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.075782 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.076560 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.076783 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.076862 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.079606 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.65:35685: BlockingRecv error: recv got EOF from 127.23.88.65:35685 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":66,"threads_started":1}
Aug 05 23:16:15 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
Aug 05 23:16:15 dist-test-slave-vkjp krb5kdc[32466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.88.66@KRBTEST.COM for oryx/127.23.88.126@KRBTEST.COM, Server not found in Kerberos database
I20260805 23:16:15.283357 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:15.277737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42183 (local address 127.23.88.66:39083)
0805 23:16:15.278054 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:15.278069 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:15.278171 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:15.278444 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:15.278446 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:15.278629 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:15.278727 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.278732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.279568 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.279571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.280142 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.280150 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.280611 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.280613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.280677 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.281258 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:15.281270 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:15.283193 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.88.126:42183: Server oryx/127.23.88.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":79,"threads_started":1}
W20260805 23:16:15.283517 1123 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:15.277852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:39083 (local address 127.23.88.126:42183)
0805 23:16:15.278068 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:15.278072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:15.278127 (+ 55us) server_negotiation.cc:407] Connection header received
0805 23:16:15.278257 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:15.278259 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:15.278304 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:15.278402 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:15.278838 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.279457 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.280256 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.280508 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.280592 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.283393 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.88.66:39083: BlockingRecv error: recv got EOF from 127.23.88.66:39083 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
I20260805 23:16:15.342573 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 618
I20260805 23:16:15.351203 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 776
I20260805 23:16:15.359272 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 916
I20260805 23:16:15.367354 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 543
2026-08-05T23:16:15Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18137 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-05T23:16:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:16:15Z Disabled control of system clock
I20260805 23:16:15.399189 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:38633
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:38633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260805 23:16:15.509341 1128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:15.509598 1128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:15.509632 1128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:15.513634 1128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:16:15.513689 1128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:15.513705 1128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:15.513765 1128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:16:15.513796 1128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:16:15.518597 1128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:38633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:38633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:15.519714 1128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:15.521100 1128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:15.529675 1133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:15.529678 1128 server_base.cc:1034] running on GCE node
W20260805 23:16:15.529747 1136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:15.529711 1134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:15.530628 1128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:15.531772 1128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:15.532948 1128 hybrid_clock.cc:648] HybridClock initialized: now 1785971775532921 us; error 36 us; skew 500 ppm
I20260805 23:16:15.534915 1128 webserver.cc:460] Webserver started at http://127.23.88.126:46415/ using document root <none> and password file <none>
I20260805 23:16:15.535564 1128 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:15.535634 1128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:15.535892 1128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:15.538002 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "655c36cdf4754a448b693162170611ce"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "66ffce78cecb619f4143a20aaf1a9686"
server_key_iv: "c63145d73885f8783a715e38829c9d59"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.538605 1128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "655c36cdf4754a448b693162170611ce"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "66ffce78cecb619f4143a20aaf1a9686"
server_key_iv: "c63145d73885f8783a715e38829c9d59"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.542466 1128 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260805 23:16:15.545068 1142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.546298 1128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:16:15.546424 1128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "655c36cdf4754a448b693162170611ce"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "66ffce78cecb619f4143a20aaf1a9686"
server_key_iv: "c63145d73885f8783a715e38829c9d59"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.546533 1128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:15.564352 1128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:15.565079 1128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:15.565311 1128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:15.572395 1128 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:38633
I20260805 23:16:15.572404 1197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:38633 every 8 connection(s)
I20260805 23:16:15.573377 1128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:16:15.575235 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1128
I20260805 23:16:15.575363 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:16:15.575688 23905 external_mini_cluster.cc:1428] Setting key 4cd5e452e4e14bb56b6988208530bcac
I20260805 23:16:15.577098 1198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:15.582796 1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: Bootstrap starting.
I20260805 23:16:15.585062 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:15.577119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59318 (local address 127.23.88.126:38633)
0805 23:16:15.577738 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:15.577753 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:15.577788 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:16:15.578481 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:15.578496 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:15.578880 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:15.579270 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:15.579701 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.580657 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.581547 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.582023 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.582193 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.583198 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:15.583204 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:15.583216 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:15.583599 (+ 383us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:15.583653 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:15.583673 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:15.583780 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:15.583907 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 23:16:15.584168 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":131,"threads_started":1}
I20260805 23:16:15.585105 1198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:15.586114 1198 log.cc:824] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:15.588810 1198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: No bootstrap required, opened a new log
I20260805 23:16:15.594357 1198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } }
I20260805 23:16:15.594776 1198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:15.594875 1198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 655c36cdf4754a448b693162170611ce, State: Initialized, Role: FOLLOWER
I20260805 23:16:15.595479 1198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } }
I20260805 23:16:15.595714 1198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:16:15.595808 1198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:16:15.595955 1198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:15.597271 1198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } }
I20260805 23:16:15.597594 1198 leader_election.cc:302] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 655c36cdf4754a448b693162170611ce; no voters:
I20260805 23:16:15.597887 1198 leader_election.cc:288] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:16:15.598083 1203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:16:15.598374 1203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [term 1 LEADER]: Becoming Leader. State: Replica: 655c36cdf4754a448b693162170611ce, State: Running, Role: LEADER
I20260805 23:16:15.598714 1203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } }
I20260805 23:16:15.599299 1198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:16:15.600746 1205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 655c36cdf4754a448b693162170611ce. Latest consensus state: current_term: 1 leader_uuid: "655c36cdf4754a448b693162170611ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } } }
I20260805 23:16:15.600761 1204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "655c36cdf4754a448b693162170611ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "655c36cdf4754a448b693162170611ce" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 38633 } } }
I20260805 23:16:15.600890 1205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [sys.catalog]: This master's current role is: LEADER
I20260805 23:16:15.600896 1204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce [sys.catalog]: This master's current role is: LEADER
W20260805 23:16:15.604202 1220 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:16:15.604278 1220 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:16:15.604354 1215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:16:15.605049 1215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:16:15.610788 1215 catalog_manager.cc:1347] Generated new cluster ID: e701cb9482894053812f047b4e7ea7f5
I20260805 23:16:15.610939 1215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:16:15.617563 1215 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:16:15.618786 1215 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:16:15.626176 1215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 655c36cdf4754a448b693162170611ce: Generated new TSK 0
I20260805 23:16:15.626992 1215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:16:15.630715 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--builtin_ntp_servers=127.23.88.84:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260805 23:16:15.744123 1224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:15.744371 1224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:15.744410 1224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:15.748421 1224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:15.748476 1224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:15.748557 1224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:16:15.753444 1224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:15.754521 1224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:15.755932 1224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:15.765303 1233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:15.765321 1229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:15.765290 1231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:15.765456 1224 server_base.cc:1034] running on GCE node
I20260805 23:16:15.766093 1224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:15.766700 1224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:15.767899 1224 hybrid_clock.cc:648] HybridClock initialized: now 1785971775767885 us; error 23 us; skew 500 ppm
I20260805 23:16:15.769716 1224 webserver.cc:460] Webserver started at http://127.23.88.65:35461/ using document root <none> and password file <none>
I20260805 23:16:15.770287 1224 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:15.770345 1224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:15.770565 1224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:15.772616 1224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "9338c8edb24e49d8b3c1f28f877cd12d"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "1729a08d45465c3b9eef276c3aeb7436"
server_key_iv: "324c50208f6d26f34b625f1db6642b8c"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.773185 1224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "9338c8edb24e49d8b3c1f28f877cd12d"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "1729a08d45465c3b9eef276c3aeb7436"
server_key_iv: "324c50208f6d26f34b625f1db6642b8c"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.776789 1224 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260805 23:16:15.779188 1240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.780351 1224 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:16:15.780465 1224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "9338c8edb24e49d8b3c1f28f877cd12d"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "1729a08d45465c3b9eef276c3aeb7436"
server_key_iv: "324c50208f6d26f34b625f1db6642b8c"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.780565 1224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:15.798913 1224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:15.799780 1224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:15.799980 1224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:15.800719 1224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:15.801697 1224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:15.801744 1224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.801817 1224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:15.801848 1224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.810634 1224 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:33117
I20260805 23:16:15.810665 1361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:33117 every 8 connection(s)
I20260805 23:16:15.811570 1224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:16:15.817826 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1224
I20260805 23:16:15.817926 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:16:15.818243 23905 external_mini_cluster.cc:1428] Setting key 3d038aa76f6c7611b4c50d4610c15e1c
I20260805 23:16:15.820362 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--builtin_ntp_servers=127.23.88.84:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260805 23:16:15.821173 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:15.813332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40705 (local address 127.23.88.126:38633)
0805 23:16:15.813498 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:15.813501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:15.814494 (+ 993us) server_negotiation.cc:407] Connection header received
0805 23:16:15.815960 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:15.815963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:15.816032 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:15.816137 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:15.817018 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.817582 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.818962 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.819266 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.819370 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.820627 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:15.820634 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:15.820639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:15.820656 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:15.820681 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:15.820693 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:15.820741 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:15.821004 (+ 263us) server_negotiation.cc:299] Negotiation successful
0805 23:16:15.821063 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 23:16:15.822285 1364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:15.813676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:38633 (local address 127.23.88.65:40705)
0805 23:16:15.814326 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:15.814382 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:15.815746 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:15.816309 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:15.816316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:15.816402 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:15.816877 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.816889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.817718 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.817720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.818837 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:15.818848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:15.819419 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:15.819422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:15.819647 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:15.820304 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:15.820306 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:15.820358 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:15.820441 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:15.820815 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:15.820827 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:15.820836 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:16:15.820973 (+ 137us) client_negotiation.cc:240] Negotiation successful
0805 23:16:15.821288 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":126,"threads_started":1}
I20260805 23:16:15.823551 1362 heartbeater.cc:347] Connected to a master server at 127.23.88.126:38633
I20260805 23:16:15.823799 1362 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:15.824404 1362 heartbeater.cc:511] Master 127.23.88.126:38633 requested a full tablet report, sending...
I20260805 23:16:15.825950 1159 ts_manager.cc:194] Registered new tserver with Master: 9338c8edb24e49d8b3c1f28f877cd12d (127.23.88.65:33117)
I20260805 23:16:15.827075 1159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.65:40705
W20260805 23:16:15.931412 1365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:15.931671 1365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:15.931733 1365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:15.935806 1365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:15.935861 1365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:15.935948 1365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:16:15.940732 1365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:15.942116 1365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:15.943614 1365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:15.953302 1370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:15.953302 1371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:15.953496 1365 server_base.cc:1034] running on GCE node
W20260805 23:16:15.953377 1373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:15.954006 1365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:15.954694 1365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:15.955904 1365 hybrid_clock.cc:648] HybridClock initialized: now 1785971775955866 us; error 52 us; skew 500 ppm
I20260805 23:16:15.957729 1365 webserver.cc:460] Webserver started at http://127.23.88.66:38451/ using document root <none> and password file <none>
I20260805 23:16:15.958339 1365 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:15.958436 1365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:15.958665 1365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:15.960807 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "d8a493891c29484098df7f0aa4d0f316"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "b1aa9afbf94f8389f541da5bfed51b81"
server_key_iv: "bbe8f58e563d551bdbffc3f4d2cd54e2"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.961411 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "d8a493891c29484098df7f0aa4d0f316"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "b1aa9afbf94f8389f541da5bfed51b81"
server_key_iv: "bbe8f58e563d551bdbffc3f4d2cd54e2"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.965134 1365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260805 23:16:15.967689 1379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.968922 1365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:16:15.969062 1365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "d8a493891c29484098df7f0aa4d0f316"
format_stamp: "Formatted at 2026-08-05 23:16:15 on dist-test-slave-vkjp"
server_key: "b1aa9afbf94f8389f541da5bfed51b81"
server_key_iv: "bbe8f58e563d551bdbffc3f4d2cd54e2"
server_key_version: "encryptionkey@0"
I20260805 23:16:15.969182 1365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:15.986299 1365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:15.987031 1365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:15.987268 1365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:15.988075 1365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:15.989120 1365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:15.989198 1365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.989282 1365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:15.989334 1365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:15.999310 1365 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:45991
I20260805 23:16:15.999325 1497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:45991 every 8 connection(s)
I20260805 23:16:16.000495 1365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:16:16.007275 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1365
I20260805 23:16:16.007436 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:16:16.007725 23905 external_mini_cluster.cc:1428] Setting key 9b80b0d1d365a9a3df6bf071d4ff31ab
I20260805 23:16:16.009819 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--builtin_ntp_servers=127.23.88.84:44243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260805 23:16:16.010186 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.002394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:44387 (local address 127.23.88.126:38633)
0805 23:16:16.002533 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.002536 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.003674 (+ 1138us) server_negotiation.cc:407] Connection header received
0805 23:16:16.005084 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.005087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.005149 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.005251 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.006115 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.006679 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.008018 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.008261 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.008364 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:16.009664 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:16.009668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:16.009671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:16.009684 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:16.009705 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:16.009718 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:16.009794 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:16.010004 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 23:16:16.010071 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:16:16.011466 1500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.002817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:38633 (local address 127.23.88.66:44387)
0805 23:16:16.003527 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:16.003568 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:16.004873 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:16.005423 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:16.005429 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:16.005509 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:16.005976 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:16.005988 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.006807 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.006811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:16.007899 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:16.007910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.008419 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.008422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:16.008651 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:16.009401 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:16.009403 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:16.009444 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:16.009509 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:16.009796 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:16.009808 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:16.009815 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:16:16.010377 (+ 562us) client_negotiation.cc:240] Negotiation successful
0805 23:16:16.010633 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":117,"threads_started":1}
I20260805 23:16:16.012656 1498 heartbeater.cc:347] Connected to a master server at 127.23.88.126:38633
I20260805 23:16:16.012887 1498 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:16.013406 1498 heartbeater.cc:511] Master 127.23.88.126:38633 requested a full tablet report, sending...
I20260805 23:16:16.014477 1159 ts_manager.cc:194] Registered new tserver with Master: d8a493891c29484098df7f0aa4d0f316 (127.23.88.66:45991)
I20260805 23:16:16.015389 1159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.66:44387
W20260805 23:16:16.120450 1501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:16.120699 1501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:16.120738 1501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:16.124723 1501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:16.124784 1501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:16.124837 1501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:16:16.129555 1501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:38633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:16.130527 1501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:16.131803 1501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:16.141095 1509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.141158 1501 server_base.cc:1034] running on GCE node
W20260805 23:16:16.141436 1507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:16.141794 1506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.142212 1501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:16.142858 1501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:16.144038 1501 hybrid_clock.cc:648] HybridClock initialized: now 1785971776144000 us; error 50 us; skew 500 ppm
I20260805 23:16:16.145910 1501 webserver.cc:460] Webserver started at http://127.23.88.67:44491/ using document root <none> and password file <none>
I20260805 23:16:16.146510 1501 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:16.146598 1501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:16.146823 1501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:16.148952 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "9ec7474c422647c899370f552661e39e"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "c78f536a0b695d6e5bf25e9c1c48b5f9"
server_key_iv: "4380f9b17a32756ed0f6f21f09614a69"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.149595 1501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "9ec7474c422647c899370f552661e39e"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "c78f536a0b695d6e5bf25e9c1c48b5f9"
server_key_iv: "4380f9b17a32756ed0f6f21f09614a69"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.153442 1501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:16:16.156062 1516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.157234 1501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260805 23:16:16.157369 1501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "9ec7474c422647c899370f552661e39e"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "c78f536a0b695d6e5bf25e9c1c48b5f9"
server_key_iv: "4380f9b17a32756ed0f6f21f09614a69"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.157502 1501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:16.178162 1501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:16.178850 1501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:16.179045 1501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:16.179685 1501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:16.180640 1501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:16.180708 1501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.180776 1501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:16.180823 1501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.189519 1501 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35077
I20260805 23:16:16.189544 1636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35077 every 8 connection(s)
I20260805 23:16:16.190382 1501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:16:16.196419 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1501
I20260805 23:16:16.196511 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:16:16.196786 23905 external_mini_cluster.cc:1428] Setting key eda579402143774471d874b636629fd3
I20260805 23:16:16.199784 1201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.192091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:36385 (local address 127.23.88.126:38633)
0805 23:16:16.192218 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.192221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.193205 (+ 984us) server_negotiation.cc:407] Connection header received
0805 23:16:16.194582 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.194585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.194646 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.194745 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.195688 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.196209 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.197650 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.197957 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.198079 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:16.199309 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:16.199312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:16.199314 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:16.199325 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:16.199344 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:16.199352 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:16.199451 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:16.199620 (+ 169us) server_negotiation.cc:299] Negotiation successful
0805 23:16:16.199672 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:16:16.200729 1639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.192445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:38633 (local address 127.23.88.67:36385)
0805 23:16:16.193063 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:16.193102 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:16.194380 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:16.194915 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:16.194922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:16.194999 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:16.195546 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:16.195558 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.196336 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.196344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:16.197508 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:16.197519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:16.198147 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:16.198152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:16.198339 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:16.199020 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:16.199022 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:16.199061 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:16.199177 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:16.199465 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:16.199473 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:16.199477 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:16:16.199610 (+ 133us) client_negotiation.cc:240] Negotiation successful
0805 23:16:16.199842 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":112,"threads_started":1}
I20260805 23:16:16.201820 1637 heartbeater.cc:347] Connected to a master server at 127.23.88.126:38633
I20260805 23:16:16.202024 1637 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:16.202477 1637 heartbeater.cc:511] Master 127.23.88.126:38633 requested a full tablet report, sending...
I20260805 23:16:16.203454 1159 ts_manager.cc:194] Registered new tserver with Master: 9ec7474c422647c899370f552661e39e (127.23.88.67:35077)
I20260805 23:16:16.203972 1159 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.67:36385
I20260805 23:16:16.210521 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:16:16.213321 1201 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:16.212446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59328 (local address 127.23.88.126:38633)
0805 23:16:16.212560 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.212564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.212770 (+ 206us) server_negotiation.cc:407] Connection header received
0805 23:16:16.212882 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.212884 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.212922 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.212989 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.213187 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59328: BlockingRecv error: recv got EOF from 127.0.0.1:59328 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:16:16.213757 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1224
I20260805 23:16:16.223430 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1365
I20260805 23:16:16.229534 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1501
I20260805 23:16:16.235152 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1128
2026-08-05T23:16:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (865 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-05T23:16:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:16:16Z Disabled control of system clock
I20260805 23:16:16.265146 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33929
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:33929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:16:16.374542 1648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:16.374822 1648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:16.374861 1648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:16.378844 1648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:16:16.378903 1648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:16.378919 1648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:16.378935 1648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:16:16.378950 1648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:16:16.383773 1648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:33929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:33929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:16.385003 1648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:16.386313 1648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:16.394382 1656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.394544 1648 server_base.cc:1034] running on GCE node
W20260805 23:16:16.394447 1653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:16.394390 1654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.395191 1648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:16.396225 1648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:16.397395 1648 hybrid_clock.cc:648] HybridClock initialized: now 1785971776397371 us; error 41 us; skew 500 ppm
I20260805 23:16:16.399235 1648 webserver.cc:460] Webserver started at http://127.23.88.126:39935/ using document root <none> and password file <none>
I20260805 23:16:16.399848 1648 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:16.399928 1648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:16.400132 1648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:16.402073 1648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "de901f6f6a754e84b4178683734d0a84"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "f7191552c944b81ff987773a007995a9"
server_key_iv: "313e92f725a141a921a69eb09d1fd740"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.402581 1648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "de901f6f6a754e84b4178683734d0a84"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "f7191552c944b81ff987773a007995a9"
server_key_iv: "313e92f725a141a921a69eb09d1fd740"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.406096 1648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.006s
I20260805 23:16:16.408478 1662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.409703 1648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:16:16.409819 1648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "de901f6f6a754e84b4178683734d0a84"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "f7191552c944b81ff987773a007995a9"
server_key_iv: "313e92f725a141a921a69eb09d1fd740"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.409924 1648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:16.414354 1648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:16.414897 1648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:16.415035 1648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:16.421962 1648 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:33929
I20260805 23:16:16.421967 1716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:33929 every 8 connection(s)
I20260805 23:16:16.422907 1648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:16:16.426093 1717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:16.430560 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1648
I20260805 23:16:16.430657 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:16:16.430945 23905 external_mini_cluster.cc:1428] Setting key dd333f78e36e9235d3ad5d102a53bf83
I20260805 23:16:16.431798 1717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: Bootstrap starting.
I20260805 23:16:16.434077 1717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:16.435009 1717 log.cc:824] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:16.436343 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.432297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.23.88.126:33929)
0805 23:16:16.432891 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.432907 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.432943 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:16:16.433569 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.433585 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.433913 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.434209 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.435010 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:16.435014 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:16.435024 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:16.435034 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:16.435126 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:16.435154 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:16.435249 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:16.435377 (+ 128us) server_negotiation.cc:299] Negotiation successful
0805 23:16:16.435645 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":133,"threads_started":1}
I20260805 23:16:16.437000 1717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: No bootstrap required, opened a new log
I20260805 23:16:16.441049 1717 raft_consensus.cc:348] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } }
I20260805 23:16:16.441283 1717 raft_consensus.cc:374] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:16.441360 1717 raft_consensus.cc:729] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de901f6f6a754e84b4178683734d0a84, State: Initialized, Role: FOLLOWER
I20260805 23:16:16.441753 1717 consensus_queue.cc:260] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } }
I20260805 23:16:16.441905 1717 raft_consensus.cc:388] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:16:16.441982 1717 raft_consensus.cc:482] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:16:16.442080 1717 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:16.442951 1717 raft_consensus.cc:504] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } }
I20260805 23:16:16.443300 1717 leader_election.cc:302] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de901f6f6a754e84b4178683734d0a84; no voters:
I20260805 23:16:16.443588 1717 leader_election.cc:288] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:16:16.443691 1722 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:16:16.443919 1722 raft_consensus.cc:686] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [term 1 LEADER]: Becoming Leader. State: Replica: de901f6f6a754e84b4178683734d0a84, State: Running, Role: LEADER
I20260805 23:16:16.444394 1722 consensus_queue.cc:237] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } }
I20260805 23:16:16.444870 1717 sys_catalog.cc:564] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:16:16.446081 1723 sys_catalog.cc:455] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de901f6f6a754e84b4178683734d0a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } } }
I20260805 23:16:16.446117 1724 sys_catalog.cc:455] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de901f6f6a754e84b4178683734d0a84. Latest consensus state: current_term: 1 leader_uuid: "de901f6f6a754e84b4178683734d0a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de901f6f6a754e84b4178683734d0a84" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 33929 } } }
I20260805 23:16:16.446213 1723 sys_catalog.cc:458] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [sys.catalog]: This master's current role is: LEADER
I20260805 23:16:16.446213 1724 sys_catalog.cc:458] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84 [sys.catalog]: This master's current role is: LEADER
W20260805 23:16:16.449542 1740 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:16:16.449606 1740 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:16:16.449699 1732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:16:16.450632 1732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:16:16.456010 1732 catalog_manager.cc:1347] Generated new cluster ID: 8aacc63f3b304f588741de91fa358431
I20260805 23:16:16.456072 1732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:16:16.487927 1732 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:16:16.488289 1732 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:16:16.497301 1732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de901f6f6a754e84b4178683734d0a84: Generated new TSK 0
I20260805 23:16:16.498057 1732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:16:16.506354 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--builtin_ntp_servers=127.23.88.84:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:16:16.640225 1743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:16.640501 1743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:16.640543 1743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:16.644539 1743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:16.644615 1743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:16.644675 1743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:16:16.649446 1743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:16.650453 1743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:16.651837 1743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:16.661561 1752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.661725 1743 server_base.cc:1034] running on GCE node
W20260805 23:16:16.661661 1749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:16.661566 1750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.662266 1743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:16.662956 1743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:16.664129 1743 hybrid_clock.cc:648] HybridClock initialized: now 1785971776664101 us; error 41 us; skew 500 ppm
I20260805 23:16:16.666085 1743 webserver.cc:460] Webserver started at http://127.23.88.65:41869/ using document root <none> and password file <none>
I20260805 23:16:16.666666 1743 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:16.666733 1743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:16.666955 1743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:16.669123 1743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "ee64a6705c2a48a19a58041d0c595891"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "9e6b9d1a4fcce767171973ad296df758"
server_key_iv: "7fe2d8801f931656787b4d31fbf9399d"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.669752 1743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "ee64a6705c2a48a19a58041d0c595891"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "9e6b9d1a4fcce767171973ad296df758"
server_key_iv: "7fe2d8801f931656787b4d31fbf9399d"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.673624 1743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:16:16.676170 1759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.677294 1743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:16:16.677428 1743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ee64a6705c2a48a19a58041d0c595891"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "9e6b9d1a4fcce767171973ad296df758"
server_key_iv: "7fe2d8801f931656787b4d31fbf9399d"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.677536 1743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:16.682008 1743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:16.682577 1743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:16.682745 1743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:16.683336 1743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:16.684212 1743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:16.684262 1743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.684301 1743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:16.684355 1743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.693042 1743 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:36793
I20260805 23:16:16.693061 1878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:36793 every 8 connection(s)
I20260805 23:16:16.693944 1743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:16:16.699990 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.695727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:41383 (local address 127.23.88.126:33929)
0805 23:16:16.695859 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.695863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.696909 (+ 1046us) server_negotiation.cc:407] Connection header received
0805 23:16:16.698169 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.698171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.698211 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.698283 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.699450 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:16.699453 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:16.699455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:16.699481 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:16.699509 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:16.699519 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:16.699623 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:16.699747 (+ 124us) server_negotiation.cc:299] Negotiation successful
0805 23:16:16.699819 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:16:16.700809 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.696107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33929 (local address 127.23.88.65:41383)
0805 23:16:16.696756 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:16.696800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:16.697989 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:16.698416 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:16.698422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:16.698507 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:16.699147 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:16.699150 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:16.699197 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:16.699269 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:16.699609 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:16.699616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:16.699620 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:16:16.699760 (+ 140us) client_negotiation.cc:240] Negotiation successful
0805 23:16:16.700000 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":116,"threads_started":1}
I20260805 23:16:16.702044 1879 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33929
I20260805 23:16:16.702243 1879 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:16.702598 1879 heartbeater.cc:511] Master 127.23.88.126:33929 requested a full tablet report, sending...
I20260805 23:16:16.703644 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1743
I20260805 23:16:16.703755 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:16:16.704025 23905 external_mini_cluster.cc:1428] Setting key b441b73065e6cd4d3d3359870347dd72
I20260805 23:16:16.704123 1679 ts_manager.cc:194] Registered new tserver with Master: ee64a6705c2a48a19a58041d0c595891 (127.23.88.65:36793)
I20260805 23:16:16.705931 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--builtin_ntp_servers=127.23.88.84:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:16:16.817472 1882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:16.817749 1882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:16.817824 1882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:16.822155 1882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:16.822227 1882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:16.822340 1882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:16:16.827198 1882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:16.828307 1882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:16.829732 1882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:16.839466 1887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.839570 1882 server_base.cc:1034] running on GCE node
W20260805 23:16:16.839519 1889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:16.839509 1891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:16.840076 1882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:16.840798 1882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:16.842008 1882 hybrid_clock.cc:648] HybridClock initialized: now 1785971776841993 us; error 28 us; skew 500 ppm
I20260805 23:16:16.843971 1882 webserver.cc:460] Webserver started at http://127.23.88.66:41407/ using document root <none> and password file <none>
I20260805 23:16:16.844547 1882 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:16.844637 1882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:16.844956 1882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:16.847230 1882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "b8f8efb3ac424467a827c59d99224b81"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "be30b27e49312a5ca25da11cdd7fb00c"
server_key_iv: "4c597aa55885586926eb207472ba13fa"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.847786 1882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "b8f8efb3ac424467a827c59d99224b81"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "be30b27e49312a5ca25da11cdd7fb00c"
server_key_iv: "4c597aa55885586926eb207472ba13fa"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.851409 1882 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:16:16.854003 1898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.855285 1882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:16:16.855400 1882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "b8f8efb3ac424467a827c59d99224b81"
format_stamp: "Formatted at 2026-08-05 23:16:16 on dist-test-slave-vkjp"
server_key: "be30b27e49312a5ca25da11cdd7fb00c"
server_key_iv: "4c597aa55885586926eb207472ba13fa"
server_key_version: "encryptionkey@0"
I20260805 23:16:16.855510 1882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:16.860047 1882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:16.860638 1882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:16.860808 1882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:16.861348 1882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:16.862273 1882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:16.862319 1882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.862380 1882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:16.862418 1882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:16.871274 1882 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:41139
I20260805 23:16:16.871340 2016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:41139 every 8 connection(s)
I20260805 23:16:16.872239 1882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:16:16.878326 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.874044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:45579 (local address 127.23.88.126:33929)
0805 23:16:16.874189 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:16.874192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:16.875270 (+ 1078us) server_negotiation.cc:407] Connection header received
0805 23:16:16.876641 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:16.876645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:16.876693 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:16.876797 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:16.877851 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:16.877855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:16.877858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:16.877871 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:16.877896 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:16.877904 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:16.877999 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:16.878143 (+ 144us) server_negotiation.cc:299] Negotiation successful
0805 23:16:16.878189 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:16:16.879284 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:16.874422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33929 (local address 127.23.88.66:45579)
0805 23:16:16.875071 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:16.875169 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:16.876430 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:16.876973 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:16.876980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:16.877064 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:16.877593 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:16.877595 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:16.877635 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:16.877698 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:16.877992 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:16.878001 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:16.878007 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:16:16.878185 (+ 178us) client_negotiation.cc:240] Negotiation successful
0805 23:16:16.878419 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":116,"threads_started":1}
I20260805 23:16:16.880434 2017 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33929
I20260805 23:16:16.880656 2017 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:16.881099 2017 heartbeater.cc:511] Master 127.23.88.126:33929 requested a full tablet report, sending...
I20260805 23:16:16.882160 1678 ts_manager.cc:194] Registered new tserver with Master: b8f8efb3ac424467a827c59d99224b81 (127.23.88.66:41139)
I20260805 23:16:16.882175 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 1882
I20260805 23:16:16.882301 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:16:16.882609 23905 external_mini_cluster.cc:1428] Setting key 941a9854631b007688778b36f7559a26
I20260805 23:16:16.884456 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--builtin_ntp_servers=127.23.88.84:39073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260805 23:16:16.995257 2020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:16.995537 2020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:16.995615 2020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:16.999626 2020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:16.999697 2020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:16.999807 2020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:16:17.004562 2020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:33929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:17.005615 2020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:17.006980 2020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:17.016534 2025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:17.016605 2029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:17.016543 2027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:17.016727 2020 server_base.cc:1034] running on GCE node
I20260805 23:16:17.017266 2020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:17.017928 2020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:17.019125 2020 hybrid_clock.cc:648] HybridClock initialized: now 1785971777019104 us; error 32 us; skew 500 ppm
I20260805 23:16:17.021117 2020 webserver.cc:460] Webserver started at http://127.23.88.67:38249/ using document root <none> and password file <none>
I20260805 23:16:17.021734 2020 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:17.021802 2020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:17.022044 2020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:17.024237 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "10ab1bae87f549fa8700c5b9b87d915f"
format_stamp: "Formatted at 2026-08-05 23:16:17 on dist-test-slave-vkjp"
server_key: "3e908f04cfea3a077ac2e229db9b46aa"
server_key_iv: "5bb1d20eef152975cb7f8342481c75ea"
server_key_version: "encryptionkey@0"
I20260805 23:16:17.024816 2020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "10ab1bae87f549fa8700c5b9b87d915f"
format_stamp: "Formatted at 2026-08-05 23:16:17 on dist-test-slave-vkjp"
server_key: "3e908f04cfea3a077ac2e229db9b46aa"
server_key_iv: "5bb1d20eef152975cb7f8342481c75ea"
server_key_version: "encryptionkey@0"
I20260805 23:16:17.028654 2020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:16:17.031420 2036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:17.032727 2020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:16:17.032850 2020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "10ab1bae87f549fa8700c5b9b87d915f"
format_stamp: "Formatted at 2026-08-05 23:16:17 on dist-test-slave-vkjp"
server_key: "3e908f04cfea3a077ac2e229db9b46aa"
server_key_iv: "5bb1d20eef152975cb7f8342481c75ea"
server_key_version: "encryptionkey@0"
I20260805 23:16:17.032963 2020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:17.037954 2020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:17.038585 2020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:17.038764 2020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:17.039376 2020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:17.040298 2020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:17.040344 2020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:17.040385 2020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:17.040441 2020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:17.049916 2020 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41763
I20260805 23:16:17.049939 2158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41763 every 8 connection(s)
I20260805 23:16:17.050830 2020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:16:17.056881 1720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:17.052743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:60005 (local address 127.23.88.126:33929)
0805 23:16:17.052891 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:17.052895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:17.053956 (+ 1061us) server_negotiation.cc:407] Connection header received
0805 23:16:17.055285 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:17.055288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:17.055327 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:17.055408 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:17.056522 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:17.056524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:17.056526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:16:17.056536 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:17.056551 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:16:17.056558 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:17.056630 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:17.056719 (+ 89us) server_negotiation.cc:299] Negotiation successful
0805 23:16:17.056756 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:16:17.057734 2161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:17.053167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:33929 (local address 127.23.88.67:60005)
0805 23:16:17.053811 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:17.053851 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:17.055027 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:17.055561 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:17.055568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:17.055653 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:17.056238 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:16:17.056240 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:17.056302 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:16:17.056390 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:17.056613 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:17.056618 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:17.056621 (+ 3us) client_negotiation.cc:769] Sending connection context
0805 23:16:17.056748 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 23:16:17.056935 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":118,"threads_started":1}
I20260805 23:16:17.058902 2159 heartbeater.cc:347] Connected to a master server at 127.23.88.126:33929
I20260805 23:16:17.059190 2159 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:17.059677 2159 heartbeater.cc:511] Master 127.23.88.126:33929 requested a full tablet report, sending...
I20260805 23:16:17.060689 1678 ts_manager.cc:194] Registered new tserver with Master: 10ab1bae87f549fa8700c5b9b87d915f (127.23.88.67:41763)
I20260805 23:16:17.060930 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 2020
I20260805 23:16:17.061051 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:16:17.061398 23905 external_mini_cluster.cc:1428] Setting key 14baa52ee5c0102d50e8c803f1b16c80
I20260805 23:16:17.063306 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260805 23:16:17.066054 1720 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:16:17.065190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51060 (local address 127.23.88.126:33929)
0805 23:16:17.065335 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:17.065341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:17.065474 (+ 133us) server_negotiation.cc:407] Connection header received
0805 23:16:17.065568 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:17.065570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:17.065604 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:17.065669 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:17.065939 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51060: BlockingRecv error: recv got EOF from 127.0.0.1:51060 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:16:17.066272 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1743
I20260805 23:16:17.076164 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1882
I20260805 23:16:17.082412 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 2020
I20260805 23:16:17.090205 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 1648
2026-08-05T23:16:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (855 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:16:17 dist-test-slave-vkjp krb5kdc[2169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:16:17 dist-test-slave-vkjp krb5kdc[2169](info): set up 2 sockets
Aug 05 23:16:17 dist-test-slave-vkjp krb5kdc[2169](info): commencing operation
krb5kdc: starting...
W20260805 23:16:18.161587 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:16:18 dist-test-slave-vkjp krb5kdc[2169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:16:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:16:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:16:18.290522 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34349
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:16:18.401906 2182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:18.402242 2182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:18.402318 2182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:18.406445 2182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:16:18.406539 2182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:18.406580 2182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:18.406634 2182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:16:18.406675 2182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:16:18.412140 2182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:18.413271 2182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:18.414335 2182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:18.423197 2190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:18.423264 2182 server_base.cc:1034] running on GCE node
W20260805 23:16:18.423202 2193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:18.423295 2189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:18.424113 2182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:18.425256 2182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:18.426463 2182 hybrid_clock.cc:648] HybridClock initialized: now 1785971778426435 us; error 40 us; skew 500 ppm
Aug 05 23:16:18 dist-test-slave-vkjp krb5kdc[2169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971778, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:18.429363 2182 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:16:18.430348 2182 webserver.cc:460] Webserver started at http://127.23.88.126:42805/ using document root <none> and password file <none>
I20260805 23:16:18.430939 2182 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:18.431033 2182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:18.431288 2182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:18.433421 2182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "cb8090ce103349f5ae7baa4102fe22b0"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "11549f3c2e1a756f4958572e3b274c99"
server_key_iv: "2f63d591e0ae72ec5ff5b04d4e252e67"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.434020 2182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "cb8090ce103349f5ae7baa4102fe22b0"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "11549f3c2e1a756f4958572e3b274c99"
server_key_iv: "2f63d591e0ae72ec5ff5b04d4e252e67"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.437873 2182 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:16:18.440454 2200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:18.441726 2182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:16:18.441871 2182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "cb8090ce103349f5ae7baa4102fe22b0"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "11549f3c2e1a756f4958572e3b274c99"
server_key_iv: "2f63d591e0ae72ec5ff5b04d4e252e67"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.442005 2182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:18.474917 2182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:18.478365 2182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:18.478600 2182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:18.486498 2182 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34349
I20260805 23:16:18.486531 2256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34349 every 8 connection(s)
I20260805 23:16:18.487532 2182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:16:18.490621 2257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:18.496815 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: Bootstrap starting.
I20260805 23:16:18.497876 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 2182
I20260805 23:16:18.498013 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:16:18.498399 23905 external_mini_cluster.cc:1428] Setting key 3b7eb51604305f4563727d04110d66b3
I20260805 23:16:18.500260 2257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:18.501098 2257 log.cc:824] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:18.503489 2257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: No bootstrap required, opened a new log
Aug 05 23:16:18 dist-test-slave-vkjp krb5kdc[2169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:16:18.508728 2257 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } }
I20260805 23:16:18.509029 2257 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:18.509125 2257 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb8090ce103349f5ae7baa4102fe22b0, State: Initialized, Role: FOLLOWER
I20260805 23:16:18.509656 2257 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } }
I20260805 23:16:18.509835 2257 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:16:18.509929 2257 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:16:18.510059 2257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:18.511142 2257 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } }
I20260805 23:16:18.511531 2257 leader_election.cc:302] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb8090ce103349f5ae7baa4102fe22b0; no voters:
I20260805 23:16:18.511952 2257 leader_election.cc:288] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:16:18.512063 2262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:16:18.512341 2262 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [term 1 LEADER]: Becoming Leader. State: Replica: cb8090ce103349f5ae7baa4102fe22b0, State: Running, Role: LEADER
I20260805 23:16:18.512759 2262 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } }
I20260805 23:16:18.513477 2257 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:16:18.514653 2264 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb8090ce103349f5ae7baa4102fe22b0. Latest consensus state: current_term: 1 leader_uuid: "cb8090ce103349f5ae7baa4102fe22b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } } }
I20260805 23:16:18.514812 2264 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:16:18.514850 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:18.499725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46602 (local address 127.23.88.126:34349)
0805 23:16:18.500155 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:18.500171 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:18.500206 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:16:18.500914 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:18.500931 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:18.501326 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:18.501665 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:18.502584 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.503699 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.504910 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.505370 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.505493 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:18.508632 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:18.508638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:18.508655 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:18.508687 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:18.510662 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:18.511169 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:18.511175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:18.511186 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:18.511333 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:18.511675 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:18.511679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:18.511681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:18.512307 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:18.513168 (+ 861us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:18.513350 (+ 182us) server_negotiation.cc:299] Negotiation successful
0805 23:16:18.513695 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":136,"threads_started":1}
I20260805 23:16:18.515249 2271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:16:18.514654 2263 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb8090ce103349f5ae7baa4102fe22b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb8090ce103349f5ae7baa4102fe22b0" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34349 } } }
I20260805 23:16:18.515373 2263 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0 [sys.catalog]: This master's current role is: LEADER
I20260805 23:16:18.518493 2271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:16:18.524343 2271 catalog_manager.cc:1347] Generated new cluster ID: 1f6e25f0d0d5491f9dec848db3ab3f75
I20260805 23:16:18.524456 2271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:16:18.538214 2271 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:16:18.539362 2271 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:16:18.545462 2271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: Generated new TSK 0
I20260805 23:16:18.546167 2271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:16:18.588685 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:16:18.745995 2283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:18.746307 2283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:18.746392 2283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:18.750819 2283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:18.750910 2283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:18.751003 2283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:16:18.756529 2283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:18.757661 2283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:18.758721 2283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:18.768674 2289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:18.768639 2290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:18.769140 2283 server_base.cc:1034] running on GCE node
W20260805 23:16:18.769193 2292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:18.769599 2283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:18.770272 2283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:18.771449 2283 hybrid_clock.cc:648] HybridClock initialized: now 1785971778771415 us; error 61 us; skew 500 ppm
Aug 05 23:16:18 dist-test-slave-vkjp krb5kdc[2169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971778, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:18.774480 2283 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:16:18.775574 2283 webserver.cc:460] Webserver started at http://127.23.88.65:40355/ using document root <none> and password file <none>
I20260805 23:16:18.776158 2283 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:18.776248 2283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:18.776455 2283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:18.778535 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "02149c1b0f7341f4a919c584dfe5645e"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "1f015bf36a1d72f352a483e0de0c9080"
server_key_iv: "51f86b5806f76ac7d4373a188f8d7793"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.779153 2283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "02149c1b0f7341f4a919c584dfe5645e"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "1f015bf36a1d72f352a483e0de0c9080"
server_key_iv: "51f86b5806f76ac7d4373a188f8d7793"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.782922 2283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:16:18.785547 2299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:18.786955 2283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:16:18.787120 2283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "02149c1b0f7341f4a919c584dfe5645e"
format_stamp: "Formatted at 2026-08-05 23:16:18 on dist-test-slave-vkjp"
server_key: "1f015bf36a1d72f352a483e0de0c9080"
server_key_iv: "51f86b5806f76ac7d4373a188f8d7793"
server_key_version: "encryptionkey@0"
I20260805 23:16:18.787240 2283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:18.813369 2283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:18.817360 2283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:18.817596 2283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:18.818274 2283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:18.819316 2283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:18.819389 2283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:18.819463 2283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:18.819514 2283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:18.831674 2283 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:34471
I20260805 23:16:18.831768 2417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:34471 every 8 connection(s)
I20260805 23:16:18.832716 2283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:16:18.839150 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 2283
I20260805 23:16:18.839296 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:16:18.839620 23905 external_mini_cluster.cc:1428] Setting key 352b71d9403758d9788ea9caf426baaa
Aug 05 23:16:18 dist-test-slave-vkjp krb5kdc[2169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971778, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:16:18.849493 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:18.834704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:59779 (local address 127.23.88.126:34349)
0805 23:16:18.834888 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:18.834894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:18.835860 (+ 966us) server_negotiation.cc:407] Connection header received
0805 23:16:18.837441 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:18.837447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:18.837523 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:18.837654 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:18.838944 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.840122 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.841686 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.842162 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.842306 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:18.845405 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:18.845409 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:18.845411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:18.845436 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:18.847240 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:18.847821 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:18.847825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:18.847827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:18.847915 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:18.848286 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:18.848290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:18.848292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:18.848492 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:18.848608 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:18.849152 (+ 544us) server_negotiation.cc:299] Negotiation successful
0805 23:16:18.849315 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:16:18.850850 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:18.835167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34349 (local address 127.23.88.65:59779)
0805 23:16:18.835668 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:18.835726 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:18.837224 (+ 1498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:18.837820 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:18.837827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:18.838206 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:18.838726 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:18.838740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.840292 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.840298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:18.841529 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:18.841542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:18.842362 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:18.842369 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:18.842648 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:18.843500 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:18.843520 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:18.845218 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:18.847372 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:18.847379 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:18.847395 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:18.847688 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:18.848017 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:18.848020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:18.848022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:18.848177 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:18.848608 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:18.848617 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:18.848900 (+ 283us) client_negotiation.cc:769] Sending connection context
0805 23:16:18.849107 (+ 207us) client_negotiation.cc:240] Negotiation successful
0805 23:16:18.849750 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":152,"threads_started":1}
I20260805 23:16:18.852348 2418 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34349
I20260805 23:16:18.852651 2418 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:18.853339 2418 heartbeater.cc:511] Master 127.23.88.126:34349 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:16:18.855024 2217 ts_manager.cc:194] Registered new tserver with Master: 02149c1b0f7341f4a919c584dfe5645e (127.23.88.65:34471)
I20260805 23:16:18.856231 2217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:59779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:16:18.871304 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:16:18.983192 2423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:18.983515 2423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:18.983603 2423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:18.987649 2423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:18.987746 2423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:18.987838 2423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:16:18.993490 2423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:18.994956 2423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:18.996106 2423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:19.006493 2433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:19.006513 2429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:19.006493 2430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:19.006896 2423 server_base.cc:1034] running on GCE node
I20260805 23:16:19.007373 2423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:19.008057 2423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:19.009248 2423 hybrid_clock.cc:648] HybridClock initialized: now 1785971779009215 us; error 46 us; skew 500 ppm
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971779, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:19.012125 2423 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:16:19.013113 2423 webserver.cc:460] Webserver started at http://127.23.88.66:33043/ using document root <none> and password file <none>
I20260805 23:16:19.013697 2423 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:19.013788 2423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:19.014005 2423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:19.016279 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "859639f1808c49ce81eccf561b294035"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ecfb1f95d52d132012d8a69c473cc1e6"
server_key_iv: "2e150bf5238d4be8acf8570ada4ceeae"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.016865 2423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "859639f1808c49ce81eccf561b294035"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ecfb1f95d52d132012d8a69c473cc1e6"
server_key_iv: "2e150bf5238d4be8acf8570ada4ceeae"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.020654 2423 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:16:19.023067 2440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.024376 2423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:16:19.024511 2423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "859639f1808c49ce81eccf561b294035"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ecfb1f95d52d132012d8a69c473cc1e6"
server_key_iv: "2e150bf5238d4be8acf8570ada4ceeae"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.024623 2423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:19.046049 2423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:19.049418 2423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:19.049628 2423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:19.050263 2423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:19.051226 2423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:19.051293 2423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.051363 2423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:19.051416 2423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.060437 2423 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:38475
I20260805 23:16:19.060513 2557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:38475 every 8 connection(s)
I20260805 23:16:19.061379 2423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:16:19.067802 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 2423
I20260805 23:16:19.067945 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:16:19.068216 23905 external_mini_cluster.cc:1428] Setting key c6d135bfff07390a38f28cb66d16ebcc
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971779, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:16:19.076093 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.063284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51535 (local address 127.23.88.126:34349)
0805 23:16:19.063450 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.063455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.064232 (+ 777us) server_negotiation.cc:407] Connection header received
0805 23:16:19.065564 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.065566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.065614 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.065689 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:19.066829 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.067428 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.068657 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.068912 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.069020 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.071987 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:19.071990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:19.071992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:19.072018 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:19.073702 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.074223 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.074227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.074229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.074331 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.074667 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.074672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.074674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.074988 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:19.075123 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.075647 (+ 524us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.075811 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:16:19.076748 2560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.063641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34349 (local address 127.23.88.66:51535)
0805 23:16:19.064081 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.064124 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.065378 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.065853 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.065860 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.066219 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:19.066697 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.066709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.067539 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.067542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.068533 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.068542 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.069058 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.069060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.069203 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.069867 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:19.069888 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:19.071800 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:19.073816 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:19.073821 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:19.073833 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:19.074112 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:19.074432 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:19.074434 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:19.074435 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:19.074569 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:19.075137 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:19.075142 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:19.075387 (+ 245us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.075557 (+ 170us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.075817 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260805 23:16:19.078006 2558 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34349
I20260805 23:16:19.078256 2558 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:16:19.078789 2558 heartbeater.cc:511] Master 127.23.88.126:34349 requested a full tablet report, sending...
I20260805 23:16:19.079895 2217 ts_manager.cc:194] Registered new tserver with Master: 859639f1808c49ce81eccf561b294035 (127.23.88.66:38475)
I20260805 23:16:19.080454 2217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:51535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:16:19.096382 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:40629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:16:19.207819 2563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:16:19.208105 2563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:16:19.208204 2563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:16:19.212213 2563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:16:19.212292 2563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:16:19.212374 2563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:16:19.218042 2563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:40629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:16:19.219241 2563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:16:19.220307 2563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:16:19.229919 2569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:16:19.229924 2568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:19.230144 2563 server_base.cc:1034] running on GCE node
W20260805 23:16:19.229952 2571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:16:19.230665 2563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:16:19.231400 2563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:16:19.232599 2563 hybrid_clock.cc:648] HybridClock initialized: now 1785971779232563 us; error 46 us; skew 500 ppm
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971779, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:16:19.235406 2563 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:16:19.236249 2563 webserver.cc:460] Webserver started at http://127.23.88.67:45639/ using document root <none> and password file <none>
I20260805 23:16:19.236800 2563 fs_manager.cc:359] Metadata directory not provided
I20260805 23:16:19.236901 2563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:16:19.237123 2563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:16:19.239176 2563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "3dd0464951ad45dca5d18e3854225f4b"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ddb8fedee2e2d006bd63069f4c9827f4"
server_key_iv: "858cb40353b868ac1d886efdea7a54f7"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.239748 2563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "3dd0464951ad45dca5d18e3854225f4b"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ddb8fedee2e2d006bd63069f4c9827f4"
server_key_iv: "858cb40353b868ac1d886efdea7a54f7"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.243479 2563 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:16:19.246014 2578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.247298 2563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:16:19.247427 2563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "3dd0464951ad45dca5d18e3854225f4b"
format_stamp: "Formatted at 2026-08-05 23:16:19 on dist-test-slave-vkjp"
server_key: "ddb8fedee2e2d006bd63069f4c9827f4"
server_key_iv: "858cb40353b868ac1d886efdea7a54f7"
server_key_version: "encryptionkey@0"
I20260805 23:16:19.247538 2563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:16:19.264027 2563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:16:19.267221 2563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:16:19.267427 2563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:16:19.268021 2563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:16:19.268836 2563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:16:19.268906 2563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.268978 2563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:16:19.269024 2563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:16:19.277571 2563 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:35657
I20260805 23:16:19.277596 2701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:35657 every 8 connection(s)
I20260805 23:16:19.278447 2563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:16:19.282840 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 2563
I20260805 23:16:19.283012 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:16:19.283329 23905 external_mini_cluster.cc:1428] Setting key f792d4f4c8c8fa2c97492cb566b20dde
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971779, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:16:19.296473 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.280233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:46685 (local address 127.23.88.126:34349)
0805 23:16:19.280412 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.280417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.281219 (+ 802us) server_negotiation.cc:407] Connection header received
0805 23:16:19.282630 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.282634 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.282695 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.282801 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:19.284248 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.284953 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.286187 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.286409 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.286515 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.293052 (+ 6537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:19.293055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:19.293057 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:19.293082 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:19.294272 (+ 1190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.294771 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.294775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.294777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.294858 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.295249 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.295254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.295255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.295436 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:19.295570 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.296164 (+ 594us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.296312 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:16:19.297420 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.280620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34349 (local address 127.23.88.67:46685)
0805 23:16:19.281073 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.281115 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.282399 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.282985 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.282993 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.283469 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:16:19.284098 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.284118 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.285084 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.285086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.286065 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.286075 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.286553 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.286557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.286716 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.291456 (+ 4740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:16:19.291473 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:16:19.292881 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:16:19.294389 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:19.294394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:19.294408 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:19.294669 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:19.294962 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:16:19.294966 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:16:19.294968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:16:19.295152 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:16:19.295572 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:16:19.295577 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:16:19.295886 (+ 309us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.296086 (+ 200us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.296432 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
I20260805 23:16:19.298485 2702 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34349
I20260805 23:16:19.298732 2702 heartbeater.cc:464] Registering TS with master...
I20260805 23:16:19.299274 2702 heartbeater.cc:511] Master 127.23.88.126:34349 requested a full tablet report, sending...
I20260805 23:16:19.300239 2217 ts_manager.cc:194] Registered new tserver with Master: 3dd0464951ad45dca5d18e3854225f4b (127.23.88.67:35657)
I20260805 23:16:19.300796 2217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:46685
I20260805 23:16:19.308800 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:16:19.318389 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.310757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46614 (local address 127.23.88.126:34349)
0805 23:16:19.310901 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.310905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.311115 (+ 210us) server_negotiation.cc:407] Connection header received
0805 23:16:19.311271 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.311279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.311320 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.311398 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:16:19.312182 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.312698 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.313560 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.313776 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.313875 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.315392 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:16:19.315396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:16:19.315398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:16:19.315429 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:16:19.316724 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.317146 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.317149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.317151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.317232 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:16:19.317506 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:16:19.317509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:16:19.317511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:16:19.317738 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:16:19.317877 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.318110 (+ 233us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.318226 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:16:19.321118 2217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:16:19.324110 2217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:16:19.338032 2714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.332308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35657 (local address 127.0.0.1:39536)
0805 23:16:19.332671 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.332726 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.332859 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.333588 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.333595 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.333624 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:16:19.333783 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.333790 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.335377 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.335380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.336578 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.336588 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.337505 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.337508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.337614 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.337657 (+ 43us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.337766 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.337877 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260805 23:16:19.338093 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.332521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.23.88.67:35657)
0805 23:16:19.333017 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.333029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.333053 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:16:19.333137 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.333142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.333376 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.333940 (+ 564us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:16:19.334105 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.335223 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.336704 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.337400 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.337507 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.337647 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.337817 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.337915 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":178,"threads_started":1}
I20260805 23:16:19.338779 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.333221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58812 (local address 127.23.88.65:34471)
0805 23:16:19.333616 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.333631 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.333785 (+ 154us) server_negotiation.cc:407] Connection header received
0805 23:16:19.333906 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.333910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.334079 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.334229 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:16:19.334478 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.335573 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.337147 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.337962 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.338063 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.338208 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.338544 (+ 336us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.338634 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":128,"threads_started":1}
I20260805 23:16:19.339191 2716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.333404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38475 (local address 127.0.0.1:49926)
0805 23:16:19.333994 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.334014 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.334092 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.334830 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.334833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.334858 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:16:19.334948 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.334954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.336241 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.336245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.337303 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.337310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.338740 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.338744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.338839 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.338860 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.338929 (+ 69us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.339012 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":96,"threads_started":1}
I20260805 23:16:19.339643 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.333558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.23.88.66:38475)
0805 23:16:19.334493 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.334504 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.334530 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:16:19.334586 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.334589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.334707 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.334849 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:16:19.335047 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.336103 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.337415 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.338448 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.338546 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.338673 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.339446 (+ 773us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.339524 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":831,"thread_start_us":143,"threads_started":1}
I20260805 23:16:19.339396 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.333023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:34471 (local address 127.0.0.1:58812)
0805 23:16:19.333696 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.333709 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.333800 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.334211 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.334214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.334233 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:16:19.334337 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.334347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.335755 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.335759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.337007 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.337017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.338277 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.338283 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.338379 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.338408 (+ 29us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.339189 (+ 781us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.339269 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260805 23:16:19.341331 2630 tablet_service.cc:1467] Processing CreateTablet for tablet 44928526373846c0ab97c784c577d640 (DEFAULT_TABLE table=test-table [id=776971d5caf14ed7811ec75f1c45f175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:16:19.342558 2630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44928526373846c0ab97c784c577d640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:19.342278 2349 tablet_service.cc:1467] Processing CreateTablet for tablet 44928526373846c0ab97c784c577d640 (DEFAULT_TABLE table=test-table [id=776971d5caf14ed7811ec75f1c45f175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:16:19.342574 2488 tablet_service.cc:1467] Processing CreateTablet for tablet 44928526373846c0ab97c784c577d640 (DEFAULT_TABLE table=test-table [id=776971d5caf14ed7811ec75f1c45f175]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:16:19.343381 2349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44928526373846c0ab97c784c577d640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:19.343612 2488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44928526373846c0ab97c784c577d640. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:16:19.349574 2720 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Bootstrap starting.
I20260805 23:16:19.350839 2721 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Bootstrap starting.
I20260805 23:16:19.351487 2722 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Bootstrap starting.
I20260805 23:16:19.352306 2720 tablet_bootstrap.cc:654] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:19.353288 2720 log.cc:824] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:19.353590 2721 tablet_bootstrap.cc:654] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:19.354197 2722 tablet_bootstrap.cc:654] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Neither blocks nor log segments found. Creating new log.
I20260805 23:16:19.354619 2721 log.cc:824] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:19.354990 2722 log.cc:824] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Log is configured to *not* fsync() on all Append() calls
I20260805 23:16:19.355662 2720 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: No bootstrap required, opened a new log
I20260805 23:16:19.355862 2720 ts_tablet_manager.cc:1397] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:16:19.356406 2721 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: No bootstrap required, opened a new log
I20260805 23:16:19.356592 2721 ts_tablet_manager.cc:1397] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:16:19.358976 2720 raft_consensus.cc:348] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.359270 2720 raft_consensus.cc:374] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:19.359345 2720 raft_consensus.cc:729] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02149c1b0f7341f4a919c584dfe5645e, State: Initialized, Role: FOLLOWER
I20260805 23:16:19.359297 2721 raft_consensus.cc:348] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.359599 2721 raft_consensus.cc:374] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:19.359665 2721 raft_consensus.cc:729] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dd0464951ad45dca5d18e3854225f4b, State: Initialized, Role: FOLLOWER
I20260805 23:16:19.359876 2720 consensus_queue.cc:260] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.360088 2721 consensus_queue.cc:260] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.360828 2702 heartbeater.cc:503] Master 127.23.88.126:34349 was elected leader, sending a full tablet report...
I20260805 23:16:19.361184 2721 ts_tablet_manager.cc:1428] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:16:19.361244 2722 tablet_bootstrap.cc:492] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: No bootstrap required, opened a new log
I20260805 23:16:19.361425 2722 ts_tablet_manager.cc:1397] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20260805 23:16:19.361424 2720 ts_tablet_manager.cc:1428] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:16:19.361776 2418 heartbeater.cc:503] Master 127.23.88.126:34349 was elected leader, sending a full tablet report...
I20260805 23:16:19.365373 2722 raft_consensus.cc:348] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.365689 2722 raft_consensus.cc:374] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:16:19.365754 2722 raft_consensus.cc:729] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 859639f1808c49ce81eccf561b294035, State: Initialized, Role: FOLLOWER
I20260805 23:16:19.366226 2722 consensus_queue.cc:260] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.366806 2558 heartbeater.cc:503] Master 127.23.88.126:34349 was elected leader, sending a full tablet report...
I20260805 23:16:19.367033 2722 ts_tablet_manager.cc:1428] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260805 23:16:19.529502 2704 tablet.cc:2369] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:16:19.562557 2559 tablet.cc:2369] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:16:19.584241 2419 tablet.cc:2369] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:16:19.722035 2726 raft_consensus.cc:482] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:16:19.722299 2726 raft_consensus.cc:504] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.723541 2726 leader_election.cc:288] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3dd0464951ad45dca5d18e3854225f4b (127.23.88.67:35657), 859639f1808c49ce81eccf561b294035 (127.23.88.66:38475)
I20260805 23:16:19.729032 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.723897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:35657 (local address 127.23.88.65:59455)
0805 23:16:19.724218 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.724232 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.724387 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.724700 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.724703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.724717 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:16:19.724866 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.724871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.725920 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.725923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.727398 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.727410 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.728396 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.728399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.728533 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.728633 (+ 100us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.728728 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.728819 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":96,"thread_start_us":117,"threads_started":1}
I20260805 23:16:19.729020 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.724500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:39835 (local address 127.23.88.66:38475)
0805 23:16:19.724665 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.724669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.724684 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:16:19.724851 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.724854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.724894 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.724985 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:16:19.725216 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.725929 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.727539 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.728236 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.728390 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.728447 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.728817 (+ 370us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.728894 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260805 23:16:19.729020 2715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.723995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:59455 (local address 127.23.88.67:35657)
0805 23:16:19.724143 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.724149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.724338 (+ 189us) server_negotiation.cc:407] Connection header received
0805 23:16:19.724508 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.724512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.724579 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.724670 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:16:19.724969 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.725810 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.727539 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.728247 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.728423 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.728477 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.728789 (+ 312us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.728885 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:16:19.729324 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.724333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38475 (local address 127.23.88.65:39835)
0805 23:16:19.724610 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.724621 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0805 23:16:19.724712 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:16:19.725008 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:16:19.725010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:16:19.725025 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:16:19.725110 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.725114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.726027 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.726029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.727399 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:16:19.727410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.728396 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.728399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:16:19.728530 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.728636 (+ 106us) client_negotiation.cc:769] Sending connection context
0805 23:16:19.728785 (+ 149us) client_negotiation.cc:240] Negotiation successful
0805 23:16:19.728834 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":128,"thread_start_us":99,"threads_started":1}
I20260805 23:16:19.729925 2508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44928526373846c0ab97c784c577d640" candidate_uuid: "02149c1b0f7341f4a919c584dfe5645e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "859639f1808c49ce81eccf561b294035" is_pre_election: true
I20260805 23:16:19.729907 2653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44928526373846c0ab97c784c577d640" candidate_uuid: "02149c1b0f7341f4a919c584dfe5645e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd0464951ad45dca5d18e3854225f4b" is_pre_election: true
I20260805 23:16:19.730257 2508 raft_consensus.cc:2454] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02149c1b0f7341f4a919c584dfe5645e in term 0.
I20260805 23:16:19.730257 2653 raft_consensus.cc:2454] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02149c1b0f7341f4a919c584dfe5645e in term 0.
I20260805 23:16:19.731523 2300 leader_election.cc:302] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02149c1b0f7341f4a919c584dfe5645e, 3dd0464951ad45dca5d18e3854225f4b; no voters:
I20260805 23:16:19.731860 2726 raft_consensus.cc:2781] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:16:19.731961 2726 raft_consensus.cc:482] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:16:19.732015 2726 raft_consensus.cc:3037] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:19.733167 2726 raft_consensus.cc:504] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.733520 2726 leader_election.cc:288] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [CANDIDATE]: Term 1 election: Requested vote from peers 3dd0464951ad45dca5d18e3854225f4b (127.23.88.67:35657), 859639f1808c49ce81eccf561b294035 (127.23.88.66:38475)
I20260805 23:16:19.733969 2508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44928526373846c0ab97c784c577d640" candidate_uuid: "02149c1b0f7341f4a919c584dfe5645e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "859639f1808c49ce81eccf561b294035"
I20260805 23:16:19.733980 2653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44928526373846c0ab97c784c577d640" candidate_uuid: "02149c1b0f7341f4a919c584dfe5645e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3dd0464951ad45dca5d18e3854225f4b"
I20260805 23:16:19.734143 2508 raft_consensus.cc:3037] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:19.734133 2653 raft_consensus.cc:3037] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:16:19.735301 2508 raft_consensus.cc:2454] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02149c1b0f7341f4a919c584dfe5645e in term 1.
I20260805 23:16:19.735534 2653 raft_consensus.cc:2454] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02149c1b0f7341f4a919c584dfe5645e in term 1.
I20260805 23:16:19.735589 2300 leader_election.cc:302] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02149c1b0f7341f4a919c584dfe5645e, 859639f1808c49ce81eccf561b294035; no voters:
I20260805 23:16:19.735793 2726 raft_consensus.cc:2781] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:16:19.736064 2726 raft_consensus.cc:686] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 1 LEADER]: Becoming Leader. State: Replica: 02149c1b0f7341f4a919c584dfe5645e, State: Running, Role: LEADER
I20260805 23:16:19.736462 2726 consensus_queue.cc:237] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } }
I20260805 23:16:19.739943 2217 catalog_manager.cc:5495] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e reported cstate change: term changed from 0 to 1, leader changed from <none> to 02149c1b0f7341f4a919c584dfe5645e (127.23.88.65). New cstate: current_term: 1 leader_uuid: "02149c1b0f7341f4a919c584dfe5645e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02149c1b0f7341f4a919c584dfe5645e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 34471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 } health_report { overall_health: UNKNOWN } } }
I20260805 23:16:19.788681 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.785288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.23.88.65:34471)
0805 23:16:19.785429 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:16:19.785433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:16:19.785447 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:16:19.785528 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:16:19.785530 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:16:19.785588 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:16:19.785684 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:16:19.785931 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.786510 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.787353 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:16:19.787580 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:16:19.787662 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:16:19.787923 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:16:19.788367 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:16:19.788460 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:16:19.788514 (+ 54us) server_negotiation.cc:299] Negotiation successful
0805 23:16:19.788565 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:16:19.795015 2508 raft_consensus.cc:1261] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 1 FOLLOWER]: Refusing update from remote peer 02149c1b0f7341f4a919c584dfe5645e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:16:19.795015 2653 raft_consensus.cc:1261] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 1 FOLLOWER]: Refusing update from remote peer 02149c1b0f7341f4a919c584dfe5645e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:16:19.795792 2732 consensus_queue.cc:1035] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [LEADER]: Connected to new peer: Peer: permanent_uuid: "859639f1808c49ce81eccf561b294035" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:16:19.796030 2726 consensus_queue.cc:1035] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3dd0464951ad45dca5d18e3854225f4b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:16:19.813736 2217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:16:19.813915 2217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:16:19.816601 2217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cb8090ce103349f5ae7baa4102fe22b0: Sending DeleteTablet for 3 replicas of tablet 44928526373846c0ab97c784c577d640
I20260805 23:16:19.817394 2630 tablet_service.cc:1514] Processing DeleteTablet for tablet 44928526373846c0ab97c784c577d640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:16:19 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39536
I20260805 23:16:19.817569 2349 tablet_service.cc:1514] Processing DeleteTablet for tablet 44928526373846c0ab97c784c577d640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:16:19 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:58812
I20260805 23:16:19.817927 2744 tablet_replica.cc:330] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: stopping tablet replica
I20260805 23:16:19.818126 2488 tablet_service.cc:1514] Processing DeleteTablet for tablet 44928526373846c0ab97c784c577d640 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:16:19 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:49926
I20260805 23:16:19.818506 2744 raft_consensus.cc:2229] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:16:19.818684 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 2283
I20260805 23:16:19.818850 2745 tablet_replica.cc:330] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e: stopping tablet replica
I20260805 23:16:19.818854 2746 tablet_replica.cc:330] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: stopping tablet replica
I20260805 23:16:19.818881 2744 raft_consensus.cc:2258] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:16:19.819185 2745 raft_consensus.cc:2229] T 44928526373846c0ab97c784c577d640 P 02149c1b0f7341f4a919c584dfe5645e [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:16:19.821224 2744 ts_tablet_manager.cc:1905] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:16:19.823998 2746 raft_consensus.cc:2229] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:16:19.824360 2746 raft_consensus.cc:2258] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:16:19.824363 2744 ts_tablet_manager.cc:1918] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:16:19.824470 2744 log.cc:1196] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/44928526373846c0ab97c784c577d640
I20260805 23:16:19.824802 2744 ts_tablet_manager.cc:1939] T 44928526373846c0ab97c784c577d640 P 3dd0464951ad45dca5d18e3854225f4b: Deleting consensus metadata
I20260805 23:16:19.825910 2746 ts_tablet_manager.cc:1905] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:16:19.826108 2201 catalog_manager.cc:4841] TS 3dd0464951ad45dca5d18e3854225f4b (127.23.88.67:35657): tablet 44928526373846c0ab97c784c577d640 (table test-table [id=776971d5caf14ed7811ec75f1c45f175]) successfully deleted
I20260805 23:16:19.828217 2746 ts_tablet_manager.cc:1918] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:16:19.828341 2746 log.cc:1196] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/44928526373846c0ab97c784c577d640
I20260805 23:16:19.828629 2746 ts_tablet_manager.cc:1939] T 44928526373846c0ab97c784c577d640 P 859639f1808c49ce81eccf561b294035: Deleting consensus metadata
W20260805 23:16:19.829866 2201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:34471 (error 108)
I20260805 23:16:19.830116 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 2423
I20260805 23:16:19.830371 2201 catalog_manager.cc:4841] TS 859639f1808c49ce81eccf561b294035 (127.23.88.66:38475): tablet 44928526373846c0ab97c784c577d640 (table test-table [id=776971d5caf14ed7811ec75f1c45f175]) successfully deleted
I20260805 23:16:19.833220 2718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:16:19.832870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:34471 (local address 127.0.0.1:58830)
0805 23:16:19.833027 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0805 23:16:19.833112 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:34471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260805 23:16:19.833393 2201 catalog_manager.cc:4584] TS 02149c1b0f7341f4a919c584dfe5645e (127.23.88.65:34471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 44928526373846c0ab97c784c577d640: Network error: Client connection negotiation failed: client connection to 127.23.88.65:34471: connect: Connection refused (error 111)
I20260805 23:16:19.836907 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 2563
I20260805 23:16:19.843065 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 2182
2026-08-05T23:16:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2754 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2750](info): set up 2 sockets
Aug 05 23:16:19 dist-test-slave-vkjp krb5kdc[2750](info): commencing operation
krb5kdc: starting...
W20260805 23:16:20.941331 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:16:21 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:16:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:16:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangeradmin_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangeradmin_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangerlookup_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangerlookup_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
I20260805 23:16:21.110487 23905 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRmBmWz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/postgres -l logfile start
2026-08-05 23:16:23.655 UTC [2782] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 23:16:23.656 UTC [2782] LOG: listening on IPv4 address "127.23.88.84", port 58207
2026-08-05 23:16:23.659 UTC [2782] LOG: listening on Unix socket "/tmp/.s.PGSQL.58207"
2026-08-05 23:16:23.694 UTC [2785] LOG: database system was shut down at 2026-08-05 23:16:22 UTC
2026-08-05 23:16:23.698 UTC [2782] LOG: database system is ready to accept connections
I20260805 23:16:24.660720 23905 mini_postgres.cc:96] Postgres bound to 58207
2026-08-05 23:16:24.667 UTC [2793] FATAL: database "slave" does not exist
127.23.88.84:58207 - accepting connections
I20260805 23:16:24.668083 23905 mini_ranger.cc:161] Starting Ranger...
I20260805 23:16:24.683385 23905 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 23:16:25.179792 23905 mini_ranger.cc:87] Created ranger Postgres database
I20260805 23:16:25.179898 23905 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin
2026-08-05 23:16:25,487 [I] DB FLAVOR :POSTGRES
2026-08-05 23:16:25,487 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:16:25,487 [I] Checking connection..
2026-08-05 23:16:25,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 23:16:25,749 [I] Checking connection passed.
2026-08-05 23:16:25,749 [I] --------- Verifying version history table ---------
2026-08-05 23:16:25,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:16:26,010 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:16:26,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:16:26,249 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:16:26,249 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 23:16:26,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 23:16:26.508 UTC [2862] WARNING: there is no transaction in progress
2026-08-05 23:16:26,514 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 23:16:26,515 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:16:26,777 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 23:16:26,777 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:16:26,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 23:16:27,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 23:16:27,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vkjp.c.gcp-upstream.internal','N') ;"
2026-08-05 23:16:27,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:16:27,790 [I] Table x_portal_user does not exist in database ranger
2026-08-05 23:16:27,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:16:28,037 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 23:16:28,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:16:28,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:16:28,528 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 23:16:28,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 23:16:29.579 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.598 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.615 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.631 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.647 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.665 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.719 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.726 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.741 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.751 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.761 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.774 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.784 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.791 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.797 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:29.804 UTC [2975] WARNING: there is no transaction in progress
2026-08-05 23:16:30,480 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 23:16:30,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 23:16:30,747 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 23:16:30,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:16:30,981 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 23:16:30,981 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:16:31,238 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 23:16:31,238 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:16:31,492 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:16:31,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vkjp.c.gcp-upstream.internal';"
2026-08-05 23:16:32,042 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 23:16:32,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 23:16:32,278 [I] DB_PATCHES have already been applied
I20260805 23:16:32.284011 23905 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRmBmWz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 23:16:32.284078 23905 mini_ranger.cc:193] Using host: 127.23.88.84
I20260805 23:16:32.287832 23905 mini_ranger.cc:227] Ranger admin URL: http://127.23.88.84:47599
Aug 05, 2026 11:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 11:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 11:16:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 11:16:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 11:16:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 23:16:33.450304 23905 mini_ranger.cc:160] Time spent starting Ranger: real 8.782s user 0.000s sys 0.010s
Aug 05, 2026 11:16:54 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 11:16:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 05, 2026 11:16:54 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 11:16:54 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 11:16:55 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 23:16:56.213482 23905 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangerkms_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/rangerkms_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.88.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.88.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 23:16:56 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971816, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 23:16:56.313318 23905 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 23:16:56.325950 23905 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 23:16:57.263883 23905 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 23:16:57.263988 23905 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms
I20260805 23:16:57.264117 23905 mini_ranger_kms.cc:227] Using postgres at 127.23.88.84:58207
2026-08-05 23:16:57,899 [I] DB FLAVOR :POSTGRES
2026-08-05 23:16:57,899 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:16:57,899 [I] Checking connection
2026-08-05 23:16:57,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 23:16:58,238 [I] connection success
2026-08-05 23:16:58,238 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 23:16:58,239 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 23:16:58,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 23:16:58,519 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 23:16:58,519 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:16:58,519 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 23:16:58,519 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:58207/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 23:16:58,790 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 23:16:59.305462 23905 mini_ranger_kms.cc:310] Created kms service
I20260805 23:16:59.522779 23905 mini_ranger_kms.cc:326] Created kudu user
I20260805 23:16:59.568612 23905 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 23:16:59 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971816, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:16:59.982801 23905 mini_ranger_kms.cc:384] Added ranger policy
I20260805 23:16:59.982954 23905 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 23:16:59.982978 23905 mini_ranger_kms.cc:240] Using host: 127.23.88.84
I20260805 23:16:59.986837 23905 mini_ranger_kms.cc:276] Ranger KMS PID: 3163
I20260805 23:16:59.986979 23905 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.88.84:56809
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 11:17:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260805 23:17:02.332816 23905 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.019s user 0.015s sys 0.593s
I20260805 23:17:02.332974 23905 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 23:17:02.333083 23905 mini_ranger_kms.cc:401] 127.23.88.84:56809/kms/v1/keys
Aug 05 23:17:02 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971822, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 05 23:17:03 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971822, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
Aug 05 23:17:03 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971822, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
Aug 05 23:17:06 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:17:06.142241 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44477
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:44477
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:56809/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:06.289749 3217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:06.290030 3217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:06.290071 3217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:06.294238 3217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:17:06.294302 3217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:06.294322 3217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:06.294339 3217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:17:06.294355 3217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:17:06.300235 3217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.88.84:56809/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:44477
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:06.301364 3217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:06.302649 3217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:06.312711 3222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:06.312800 3225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:06.312855 3223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:06.313452 3217 server_base.cc:1034] running on GCE node
I20260805 23:17:06.314091 3217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:06.315395 3217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:06.316604 3217 hybrid_clock.cc:648] HybridClock initialized: now 1785971826316582 us; error 49 us; skew 500 ppm
Aug 05 23:17:06 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971826, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:06.324150 3217 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:17:06.325346 3217 webserver.cc:460] Webserver started at http://127.23.88.126:46855/ using document root <none> and password file <none>
I20260805 23:17:06.326014 3217 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:06.326078 3217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:06.326323 3217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:06 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971826, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:06.379678 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "3440222bec8b4c09ba0dbe3038a0037c"
format_stamp: "Formatted at 2026-08-05 23:17:06 on dist-test-slave-vkjp"
server_key: "28493714284d22ed2666ceee823b6d42"
server_key_iv: "0fa17f637c10a1b696669a6c2bf68a01"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:06.380394 3217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "3440222bec8b4c09ba0dbe3038a0037c"
format_stamp: "Formatted at 2026-08-05 23:17:06 on dist-test-slave-vkjp"
server_key: "28493714284d22ed2666ceee823b6d42"
server_key_iv: "0fa17f637c10a1b696669a6c2bf68a01"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:06.384562 3217 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:17:06.400053 3232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:06.401734 3217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:17:06.401888 3217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3440222bec8b4c09ba0dbe3038a0037c"
format_stamp: "Formatted at 2026-08-05 23:17:06 on dist-test-slave-vkjp"
server_key: "28493714284d22ed2666ceee823b6d42"
server_key_iv: "0fa17f637c10a1b696669a6c2bf68a01"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:06.402029 3217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:06.434322 3217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:06.437927 3217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:06.438210 3217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:06.452509 3217 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:44477
I20260805 23:17:06.452661 3286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:44477 every 8 connection(s)
I20260805 23:17:06.453636 3217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:17:06.457231 3287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:06.461696 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 3217
I20260805 23:17:06.461849 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:17:06.462133 23905 external_mini_cluster.cc:1428] Setting key 02631d3e026708c70c4ce4c4a8114768
I20260805 23:17:06.465512 3287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: Bootstrap starting.
I20260805 23:17:06.469154 3287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:06.470438 3287 log.cc:824] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:17:06 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971826, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:06.473114 3287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: No bootstrap required, opened a new log
I20260805 23:17:06.478186 3287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } }
I20260805 23:17:06.478488 3287 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:06.478572 3287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3440222bec8b4c09ba0dbe3038a0037c, State: Initialized, Role: FOLLOWER
I20260805 23:17:06.478642 3291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:06.463752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43648 (local address 127.23.88.126:44477)
0805 23:17:06.464325 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:06.464341 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:06.464379 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:17:06.465290 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:06.465309 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:06.465750 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:06.466095 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:06.467016 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:06.468285 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:06.469176 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:06.469775 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:06.469962 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:06.472944 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:06.472950 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:06.472963 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:06.472991 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:06.475620 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:06.476086 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:06.476091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:06.476095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:06.476200 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:06.476488 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:06.476491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:06.476493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:06.476887 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:06.477091 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:06.477380 (+ 289us) server_negotiation.cc:299] Negotiation successful
0805 23:17:06.477783 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":241,"threads_started":1}
I20260805 23:17:06.479053 3287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } }
I20260805 23:17:06.479243 3287 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:17:06.479342 3287 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:17:06.479441 3287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:06.480590 3287 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } }
I20260805 23:17:06.480940 3287 leader_election.cc:302] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3440222bec8b4c09ba0dbe3038a0037c; no voters:
I20260805 23:17:06.481504 3287 leader_election.cc:288] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:17:06.481706 3293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:17:06.482103 3293 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [term 1 LEADER]: Becoming Leader. State: Replica: 3440222bec8b4c09ba0dbe3038a0037c, State: Running, Role: LEADER
I20260805 23:17:06.482475 3293 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } }
I20260805 23:17:06.482756 3287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:17:06.483716 3293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3440222bec8b4c09ba0dbe3038a0037c. Latest consensus state: current_term: 1 leader_uuid: "3440222bec8b4c09ba0dbe3038a0037c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } } }
I20260805 23:17:06.484118 3293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:06.484830 3294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3440222bec8b4c09ba0dbe3038a0037c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3440222bec8b4c09ba0dbe3038a0037c" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44477 } } }
I20260805 23:17:06.485083 3294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:06.486270 3287 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:17:06.487920 3302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:17:06.496500 3302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:17:06.504068 3302 catalog_manager.cc:1347] Generated new cluster ID: d98a43dc9c50410b82e6061fc13021f4
I20260805 23:17:06.504138 3302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:17:06.529547 3302 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:17:06.531109 3302 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:17:06.539597 3302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: Generated new TSK 0
I20260805 23:17:06.540496 3302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:17:07.532564 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:56809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:07.671065 3335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:07.671453 3335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:07.671520 3335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:07.677697 3335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:07.677798 3335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:07.677896 3335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:17:07.685976 3335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.88.84:56809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:07.687274 3335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:07.688351 3335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:07.700220 3343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:07.700486 3345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:07.700212 3342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:07.701381 3335 server_base.cc:1034] running on GCE node
I20260805 23:17:07.701854 3335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:07.702692 3335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:07.703899 3335 hybrid_clock.cc:648] HybridClock initialized: now 1785971827703876 us; error 39 us; skew 500 ppm
Aug 05 23:17:07 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:07.707983 3335 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:17:07.709229 3335 webserver.cc:460] Webserver started at http://127.23.88.65:33765/ using document root <none> and password file <none>
I20260805 23:17:07.709885 3335 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:07.709959 3335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:07.710228 3335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:07 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:07.738451 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "6cd7278488c547329dfd4d83f23f0a9e"
format_stamp: "Formatted at 2026-08-05 23:17:07 on dist-test-slave-vkjp"
server_key: "79b90dfe9e7a553c925c3e91c574e5a4"
server_key_iv: "965a1edc42940b484f5869c81b9b1bff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:07.739377 3335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "6cd7278488c547329dfd4d83f23f0a9e"
format_stamp: "Formatted at 2026-08-05 23:17:07 on dist-test-slave-vkjp"
server_key: "79b90dfe9e7a553c925c3e91c574e5a4"
server_key_iv: "965a1edc42940b484f5869c81b9b1bff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:07.748745 3335 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260805 23:17:07.774113 3352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:07.776144 3335 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.001s
I20260805 23:17:07.776381 3335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "6cd7278488c547329dfd4d83f23f0a9e"
format_stamp: "Formatted at 2026-08-05 23:17:07 on dist-test-slave-vkjp"
server_key: "79b90dfe9e7a553c925c3e91c574e5a4"
server_key_iv: "965a1edc42940b484f5869c81b9b1bff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:07.776628 3335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:07.803730 3335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:07.807929 3335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:07.808215 3335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:07.809151 3335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:07.810345 3335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:07.810467 3335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:07.813094 3335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:07.813230 3335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:07.827461 3335 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:37793
I20260805 23:17:07.828800 3335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:17:07.831604 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 3335
I20260805 23:17:07.831759 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:17:07.832101 23905 external_mini_cluster.cc:1428] Setting key 539327d4b4507f16b87614bbef5ecf8e
I20260805 23:17:07.829591 3474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:37793 every 8 connection(s)
Aug 05 23:17:07 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:07.850281 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:07.834412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:56189 (local address 127.23.88.126:44477)
0805 23:17:07.834933 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:07.834938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:07.836859 (+ 1921us) server_negotiation.cc:407] Connection header received
0805 23:17:07.836910 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:07.836912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:07.836967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:07.837060 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:07.838683 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:07.839551 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:07.841360 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:07.841727 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:07.841814 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:07.845296 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:07.845303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:07.845309 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:07.845344 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:07.847407 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:07.848163 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:07.848167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:07.848169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:07.848273 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:07.848774 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:07.848777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:07.848779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:07.849035 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:07.849117 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:07.849926 (+ 809us) server_negotiation.cc:299] Negotiation successful
0805 23:17:07.850082 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":100,"threads_started":1}
I20260805 23:17:07.852434 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:07.833284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44477 (local address 127.23.88.65:56189)
0805 23:17:07.835433 (+ 2149us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:07.835504 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:07.836553 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:07.837383 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:07.837392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:07.837782 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:07.838490 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:07.838513 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:07.839730 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:07.839734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:07.841187 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:07.841200 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:07.842054 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:07.842059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:07.842267 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:07.843028 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:07.843051 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:07.845065 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:07.847601 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:07.847608 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:07.847627 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:07.848024 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:07.848405 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:07.848408 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:07.848410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:07.848642 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:07.849209 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:07.849216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:07.849602 (+ 386us) client_negotiation.cc:769] Sending connection context
0805 23:17:07.850871 (+ 1269us) client_negotiation.cc:240] Negotiation successful
0805 23:17:07.851306 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1905,"thread_start_us":172,"threads_started":1}
I20260805 23:17:07.854254 3475 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44477
I20260805 23:17:07.854661 3475 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:17:07.863804 3475 heartbeater.cc:511] Master 127.23.88.126:44477 requested a full tablet report, sending...
I20260805 23:17:07.865947 3250 ts_manager.cc:194] Registered new tserver with Master: 6cd7278488c547329dfd4d83f23f0a9e (127.23.88.65:37793)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:17:07.867655 3250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:56189
Aug 05 23:17:07 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:17:07.900373 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:56809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:08.069376 3481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:08.070220 3481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:08.070361 3481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:08.076413 3481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:08.076632 3481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:08.076774 3481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:17:08.085160 3481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.88.84:56809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:08.086866 3481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:08.088406 3481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:08.101753 3488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:08.102142 3481 server_base.cc:1034] running on GCE node
W20260805 23:17:08.102675 3487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:08.102995 3490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:08.103579 3481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:08.104341 3481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:08.105512 3481 hybrid_clock.cc:648] HybridClock initialized: now 1785971828105448 us; error 79 us; skew 500 ppm
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:08.109198 3481 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:17:08.110253 3481 webserver.cc:460] Webserver started at http://127.23.88.66:41061/ using document root <none> and password file <none>
I20260805 23:17:08.110939 3481 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:08.111016 3481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:08.111471 3481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:08.133836 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "094469e5b501446fbbb4f56ad5bb5200"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "0ba28a3bbf10c3f6b57f396092a697ae"
server_key_iv: "9661bbb1d59176c6911e02308d2df464"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.134738 3481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "094469e5b501446fbbb4f56ad5bb5200"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "0ba28a3bbf10c3f6b57f396092a697ae"
server_key_iv: "9661bbb1d59176c6911e02308d2df464"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.140805 3481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260805 23:17:08.166025 3497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.168023 3481 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.002s
I20260805 23:17:08.168195 3481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "094469e5b501446fbbb4f56ad5bb5200"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "0ba28a3bbf10c3f6b57f396092a697ae"
server_key_iv: "9661bbb1d59176c6911e02308d2df464"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.168367 3481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:08.183418 3481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:08.187558 3481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:08.187832 3481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:08.188608 3481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:08.189728 3481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:08.189852 3481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.189961 3481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:08.190027 3481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.202664 3481 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:45653
I20260805 23:17:08.202740 3618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:45653 every 8 connection(s)
I20260805 23:17:08.203820 3481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:17:08.205835 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 3481
I20260805 23:17:08.205971 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:17:08.206288 23905 external_mini_cluster.cc:1428] Setting key 2188a011953ae9dc9f55134ab88cbd84
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:17:08.220173 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.205869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51447 (local address 127.23.88.126:44477)
0805 23:17:08.206058 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.206064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.207185 (+ 1121us) server_negotiation.cc:407] Connection header received
0805 23:17:08.208380 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.208383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.208448 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.208548 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:08.209659 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.210507 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.212166 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.212389 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.212496 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.215475 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:08.215478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:08.215480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:08.215506 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:08.217294 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.218004 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.218008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.218010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.218107 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.218603 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.218610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.218613 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.218871 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:08.218974 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.219747 (+ 773us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.219882 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260805 23:17:08.221271 3621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.206362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44477 (local address 127.23.88.66:51447)
0805 23:17:08.206953 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.207063 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.208185 (+ 1122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.208723 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.208730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.209046 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:08.209520 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.209534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.210663 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.210667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.212035 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.212049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.212556 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.212560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.212757 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.213483 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:08.213505 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:08.215279 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:08.217443 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:08.217453 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:08.217475 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:08.217863 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:08.218222 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:08.218227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:08.218233 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:08.218463 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:08.219033 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:08.219040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:08.219411 (+ 371us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.219650 (+ 239us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.220017 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:17:08.222630 3619 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44477
I20260805 23:17:08.222911 3619 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:08.223697 3619 heartbeater.cc:511] Master 127.23.88.126:44477 requested a full tablet report, sending...
I20260805 23:17:08.224994 3250 ts_manager.cc:194] Registered new tserver with Master: 094469e5b501446fbbb4f56ad5bb5200 (127.23.88.66:45653)
I20260805 23:17:08.225867 3250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:51447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:17:08.244017 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:56809/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:08.386440 3625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:08.386806 3625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:08.386929 3625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:08.391673 3625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:08.391762 3625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:08.391883 3625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:17:08.398541 3625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.88.84:56809/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:08.399977 3625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:08.401084 3625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:08.415715 3631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:08.416213 3630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:08.416272 3625 server_base.cc:1034] running on GCE node
W20260805 23:17:08.416483 3633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:08.417007 3625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:08.417857 3625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:08.425272 3625 hybrid_clock.cc:648] HybridClock initialized: now 1785971828425218 us; error 58 us; skew 500 ppm
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:08.429876 3625 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:17:08.431164 3625 webserver.cc:460] Webserver started at http://127.23.88.67:37585/ using document root <none> and password file <none>
I20260805 23:17:08.431921 3625 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:08.432025 3625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:08.432353 3625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:08.454119 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "316f3b2b7c0c4808b0b76655d38c33bf"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "8ed1e129e1e7a4d8edd140469e2c30e5"
server_key_iv: "9cfc6ccdf7c1dd8b0958fc3202049ba7"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.455149 3625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "316f3b2b7c0c4808b0b76655d38c33bf"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "8ed1e129e1e7a4d8edd140469e2c30e5"
server_key_iv: "9cfc6ccdf7c1dd8b0958fc3202049ba7"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.464948 3625 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260805 23:17:08.491636 3640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.493592 3625 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260805 23:17:08.493738 3625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "316f3b2b7c0c4808b0b76655d38c33bf"
format_stamp: "Formatted at 2026-08-05 23:17:08 on dist-test-slave-vkjp"
server_key: "8ed1e129e1e7a4d8edd140469e2c30e5"
server_key_iv: "9cfc6ccdf7c1dd8b0958fc3202049ba7"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:08.493901 3625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:08.511201 3625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:08.515033 3625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:08.515331 3625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:08.516055 3625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:08.517180 3625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:08.517300 3625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.517406 3625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:08.517477 3625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:08.541111 3625 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:42867
I20260805 23:17:08.541245 3759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:42867 every 8 connection(s)
I20260805 23:17:08.542222 3625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:17:08.543607 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 3625
I20260805 23:17:08.543753 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:17:08.544082 23905 external_mini_cluster.cc:1428] Setting key a4fbcb03cbcd8ef2c7fb6a6cb4061acf
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971828, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:08.557724 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.544682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:38085 (local address 127.23.88.126:44477)
0805 23:17:08.544868 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.544872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.546289 (+ 1417us) server_negotiation.cc:407] Connection header received
0805 23:17:08.547595 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.547599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.547656 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.547742 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:08.549065 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.549608 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.550788 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.551028 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.551172 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.553924 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:08.553927 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:08.553929 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:08.553952 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:08.555484 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.556070 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.556079 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.556080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.556154 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.556518 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.556522 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.556524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.556707 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:08.556822 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.557426 (+ 604us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.557548 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260805 23:17:08.558887 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.545196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44477 (local address 127.23.88.67:38085)
0805 23:17:08.546136 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.546190 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.547387 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.547940 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.547947 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.548347 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:08.548918 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.548934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.549740 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.549748 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.550656 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.550666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.551220 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.551224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.551403 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.552035 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:08.552051 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:08.553709 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:08.555631 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:08.555639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:08.555658 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:08.555951 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:08.556249 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:08.556253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:08.556255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:08.556416 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:08.556821 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:08.556828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:08.557162 (+ 334us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.557386 (+ 224us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.557700 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":258,"threads_started":1}
I20260805 23:17:08.560180 3760 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44477
I20260805 23:17:08.560444 3760 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:08.560961 3760 heartbeater.cc:511] Master 127.23.88.126:44477 requested a full tablet report, sending...
I20260805 23:17:08.562166 3250 ts_manager.cc:194] Registered new tserver with Master: 316f3b2b7c0c4808b0b76655d38c33bf (127.23.88.67:42867)
I20260805 23:17:08.562737 3250 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:38085
I20260805 23:17:08.571827 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:17:08.587217 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.575333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43662 (local address 127.23.88.126:44477)
0805 23:17:08.575566 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.575572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.575618 (+ 46us) server_negotiation.cc:407] Connection header received
0805 23:17:08.575702 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.575709 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.575779 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.575910 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:08.577037 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.577987 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.579256 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.579616 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.579745 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.581745 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:08.581751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:08.581754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:08.581788 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:08.584024 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.584683 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.584687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.584689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.584787 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:08.585279 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:08.585283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:08.585286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:08.585545 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:08.585702 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.585988 (+ 286us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.586126 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260805 23:17:08.591261 3250 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:17:08.596215 3250 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:17:08.619393 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.608063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:45653 (local address 127.0.0.1:39978)
0805 23:17:08.609871 (+ 1808us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.609931 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.610084 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.614564 (+ 4480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.614567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.614587 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:08.614678 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.614684 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.616233 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.616237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.617499 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.617511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.618884 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.618888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.619008 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.619033 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.619144 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.619224 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1601,"mutex_wait_us":4,"thread_start_us":119,"threads_started":1}
I20260805 23:17:08.620419 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.608249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.23.88.66:45653)
0805 23:17:08.613854 (+ 5605us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.613875 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.613913 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:17:08.614010 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.614016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.614236 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.614398 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:08.614825 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.616089 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.617689 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.618520 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.618634 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.618796 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.620119 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.620229 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5438,"thread_start_us":143,"threads_started":1}
I20260805 23:17:08.620589 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.609868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48368 (local address 127.23.88.67:42867)
0805 23:17:08.610853 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.610866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.611138 (+ 272us) server_negotiation.cc:407] Connection header received
0805 23:17:08.611372 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.611378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.611582 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.611727 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:08.612194 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.613461 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.616560 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.618258 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.618362 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.618538 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.620216 (+ 1678us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.620348 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":123,"threads_started":1}
I20260805 23:17:08.621268 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.609422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:42867 (local address 127.0.0.1:48368)
0805 23:17:08.611005 (+ 1583us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.611020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.611215 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.611896 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.611903 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.611924 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:08.612061 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.612067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.613611 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.613616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.616310 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.616324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.618656 (+ 2332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.618660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.618790 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.618834 (+ 44us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.618964 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.621092 (+ 2128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"thread_start_us":114,"threads_started":1}
I20260805 23:17:08.624151 3548 tablet_service.cc:1467] Processing CreateTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 (DEFAULT_TABLE table=test-table [id=6b6ae9cde3b745c28e00c26fe3b6dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:08.625492 3548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ced6c9edfa4f8491377edfb4e148d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:08.626384 3677 tablet_service.cc:1467] Processing CreateTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 (DEFAULT_TABLE table=test-table [id=6b6ae9cde3b745c28e00c26fe3b6dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:08.627460 3677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ced6c9edfa4f8491377edfb4e148d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:08.633069 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.610724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37793 (local address 127.0.0.1:57710)
0805 23:17:08.619893 (+ 9169us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.619910 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.619988 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.626221 (+ 6233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.626225 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.626263 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:08.626371 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.626379 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.628166 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.628170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.629601 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.629616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.632505 (+ 2889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.632510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.632629 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.632666 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.632769 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.632868 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9096,"thread_start_us":161,"threads_started":1}
I20260805 23:17:08.633821 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.611305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57710 (local address 127.23.88.65:37793)
0805 23:17:08.625482 (+ 14177us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.625491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.625522 (+ 31us) server_negotiation.cc:407] Connection header received
0805 23:17:08.625622 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.625628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.625856 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.626033 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:08.626560 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.627988 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.629795 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.632081 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.632226 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.632415 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.633480 (+ 1065us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.633606 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14039,"thread_start_us":129,"threads_started":1}
I20260805 23:17:08.637081 3780 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Bootstrap starting.
I20260805 23:17:08.637472 3403 tablet_service.cc:1467] Processing CreateTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 (DEFAULT_TABLE table=test-table [id=6b6ae9cde3b745c28e00c26fe3b6dc22]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:08.638819 3403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81ced6c9edfa4f8491377edfb4e148d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:08.641698 3780 tablet_bootstrap.cc:654] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:08.642786 3780 log.cc:824] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:08.649199 3783 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Bootstrap starting.
I20260805 23:17:08.651060 3780 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: No bootstrap required, opened a new log
I20260805 23:17:08.651341 3780 ts_tablet_manager.cc:1397] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.005s
I20260805 23:17:08.652778 3783 tablet_bootstrap.cc:654] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:08.653290 3782 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Bootstrap starting.
I20260805 23:17:08.656841 3783 log.cc:824] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:08.657612 3780 raft_consensus.cc:348] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.657893 3782 tablet_bootstrap.cc:654] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:08.657998 3780 raft_consensus.cc:374] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:08.658084 3780 raft_consensus.cc:729] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 094469e5b501446fbbb4f56ad5bb5200, State: Initialized, Role: FOLLOWER
I20260805 23:17:08.658895 3780 consensus_queue.cc:260] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.659744 3782 log.cc:824] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:08.660382 3780 ts_tablet_manager.cc:1428] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Time spent starting tablet: real 0.009s user 0.003s sys 0.004s
I20260805 23:17:08.660820 3619 heartbeater.cc:503] Master 127.23.88.126:44477 was elected leader, sending a full tablet report...
I20260805 23:17:08.663267 3782 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: No bootstrap required, opened a new log
I20260805 23:17:08.663529 3782 ts_tablet_manager.cc:1397] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260805 23:17:08.669198 3783 tablet_bootstrap.cc:492] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: No bootstrap required, opened a new log
I20260805 23:17:08.669545 3783 ts_tablet_manager.cc:1397] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Time spent bootstrapping tablet: real 0.021s user 0.000s sys 0.007s
I20260805 23:17:08.669900 3782 raft_consensus.cc:348] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.670239 3782 raft_consensus.cc:374] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:08.670321 3782 raft_consensus.cc:729] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cd7278488c547329dfd4d83f23f0a9e, State: Initialized, Role: FOLLOWER
I20260805 23:17:08.670814 3782 consensus_queue.cc:260] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.672171 3782 ts_tablet_manager.cc:1428] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260805 23:17:08.672596 3475 heartbeater.cc:503] Master 127.23.88.126:44477 was elected leader, sending a full tablet report...
I20260805 23:17:08.677687 3783 raft_consensus.cc:348] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.677963 3783 raft_consensus.cc:374] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:08.678048 3783 raft_consensus.cc:729] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 316f3b2b7c0c4808b0b76655d38c33bf, State: Initialized, Role: FOLLOWER
I20260805 23:17:08.678560 3783 consensus_queue.cc:260] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.679344 3760 heartbeater.cc:503] Master 127.23.88.126:44477 was elected leader, sending a full tablet report...
I20260805 23:17:08.681522 3783 ts_tablet_manager.cc:1428] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260805 23:17:08.709146 3620 tablet.cc:2369] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 05 23:17:08 dist-test-slave-vkjp krb5kdc[2750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971827, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
W20260805 23:17:08.795012 3761 tablet.cc:2369] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:17:08.822345 3788 raft_consensus.cc:482] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:17:08.822602 3788 raft_consensus.cc:504] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.824091 3788 leader_election.cc:288] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 094469e5b501446fbbb4f56ad5bb5200 (127.23.88.66:45653), 6cd7278488c547329dfd4d83f23f0a9e (127.23.88.65:37793)
I20260805 23:17:08.829452 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.825253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:46903 (local address 127.23.88.65:37793)
0805 23:17:08.825386 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.825390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.825402 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:17:08.825461 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.825463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.825502 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.825590 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:08.825868 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.826365 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.827836 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.828566 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.828703 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.828768 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.829205 (+ 437us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.829302 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:17:08.829458 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.824299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:45653 (local address 127.23.88.67:33447)
0805 23:17:08.824596 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.824615 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.824720 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.825508 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.825511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.825531 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:08.825844 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.825855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.826942 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.826946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.828032 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.828040 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.828980 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.828984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.829143 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.829167 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.829245 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.829322 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210}
I20260805 23:17:08.829985 3791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.824803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37793 (local address 127.23.88.67:46903)
0805 23:17:08.825199 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:08.825214 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:08.825306 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:08.825602 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:08.825605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:08.825624 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:08.825748 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.825753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.826480 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.826486 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.827693 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:08.827703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.828844 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.828848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:08.829031 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.829062 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:17:08.829609 (+ 547us) client_negotiation.cc:240] Negotiation successful
0805 23:17:08.829695 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":157,"threads_started":1}
I20260805 23:17:08.829990 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.824918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:33447 (local address 127.23.88.66:45653)
0805 23:17:08.825104 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.825110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.825129 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:17:08.825195 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.825198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.825263 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.825362 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:08.826011 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.826808 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.828144 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.828690 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.828840 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.828894 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.829751 (+ 857us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.829847 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260805 23:17:08.830629 3571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ced6c9edfa4f8491377edfb4e148d8" candidate_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094469e5b501446fbbb4f56ad5bb5200" is_pre_election: true
I20260805 23:17:08.830972 3571 raft_consensus.cc:2454] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 316f3b2b7c0c4808b0b76655d38c33bf in term 0.
I20260805 23:17:08.831596 3643 leader_election.cc:302] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094469e5b501446fbbb4f56ad5bb5200, 316f3b2b7c0c4808b0b76655d38c33bf; no voters:
I20260805 23:17:08.831921 3788 raft_consensus.cc:2781] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:17:08.832020 3788 raft_consensus.cc:482] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:17:08.832063 3788 raft_consensus.cc:3037] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 0 FOLLOWER]: Advancing to term 1
W20260805 23:17:08.833073 3476 tablet.cc:2369] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:17:08.833509 3788 raft_consensus.cc:504] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.833866 3788 leader_election.cc:288] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [CANDIDATE]: Term 1 election: Requested vote from peers 094469e5b501446fbbb4f56ad5bb5200 (127.23.88.66:45653), 6cd7278488c547329dfd4d83f23f0a9e (127.23.88.65:37793)
I20260805 23:17:08.834340 3571 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ced6c9edfa4f8491377edfb4e148d8" candidate_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "094469e5b501446fbbb4f56ad5bb5200"
I20260805 23:17:08.834482 3571 raft_consensus.cc:3037] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:08.836110 3571 raft_consensus.cc:2454] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 316f3b2b7c0c4808b0b76655d38c33bf in term 1.
I20260805 23:17:08.836560 3643 leader_election.cc:302] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 094469e5b501446fbbb4f56ad5bb5200, 316f3b2b7c0c4808b0b76655d38c33bf; no voters:
I20260805 23:17:08.836789 3788 raft_consensus.cc:2781] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:17:08.837209 3788 raft_consensus.cc:686] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 1 LEADER]: Becoming Leader. State: Replica: 316f3b2b7c0c4808b0b76655d38c33bf, State: Running, Role: LEADER
I20260805 23:17:08.837888 3788 consensus_queue.cc:237] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } }
I20260805 23:17:08.840575 3420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ced6c9edfa4f8491377edfb4e148d8" candidate_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd7278488c547329dfd4d83f23f0a9e" is_pre_election: true
I20260805 23:17:08.840662 3417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81ced6c9edfa4f8491377edfb4e148d8" candidate_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6cd7278488c547329dfd4d83f23f0a9e"
I20260805 23:17:08.840844 3417 raft_consensus.cc:3037] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:08.842890 3417 raft_consensus.cc:2454] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 316f3b2b7c0c4808b0b76655d38c33bf in term 1.
I20260805 23:17:08.843565 3250 catalog_manager.cc:5495] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 316f3b2b7c0c4808b0b76655d38c33bf (127.23.88.67). New cstate: current_term: 1 leader_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "316f3b2b7c0c4808b0b76655d38c33bf" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 42867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 } health_report { overall_health: UNKNOWN } } }
I20260805 23:17:08.925760 3777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:08.920115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.23.88.67:42867)
0805 23:17:08.920272 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:08.920277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:08.920400 (+ 123us) server_negotiation.cc:407] Connection header received
0805 23:17:08.920621 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:08.920625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:08.920701 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:08.920819 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:17:08.921367 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.922152 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.923453 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:08.923774 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:08.923873 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:08.924314 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:17:08.924911 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:17:08.925039 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:08.925392 (+ 353us) server_negotiation.cc:299] Negotiation successful
0805 23:17:08.925469 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:17:08.944588 3417 raft_consensus.cc:1261] T 81ced6c9edfa4f8491377edfb4e148d8 P 6cd7278488c547329dfd4d83f23f0a9e [term 1 FOLLOWER]: Refusing update from remote peer 316f3b2b7c0c4808b0b76655d38c33bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:17:08.945431 3788 consensus_queue.cc:1035] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6cd7278488c547329dfd4d83f23f0a9e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:17:08.953372 3571 raft_consensus.cc:1261] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 1 FOLLOWER]: Refusing update from remote peer 316f3b2b7c0c4808b0b76655d38c33bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:17:08.954252 3788 consensus_queue.cc:1035] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "094469e5b501446fbbb4f56ad5bb5200" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 45653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:17:08.988039 3249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:17:08.988262 3249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:17:08.991811 3249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3440222bec8b4c09ba0dbe3038a0037c: Sending DeleteTablet for 3 replicas of tablet 81ced6c9edfa4f8491377edfb4e148d8
I20260805 23:17:08.992885 3403 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:57710
I20260805 23:17:08.993216 3677 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:48368
I20260805 23:17:08.994009 3806 tablet_replica.cc:330] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: stopping tablet replica
I20260805 23:17:08.994316 3548 tablet_service.cc:1514] Processing DeleteTablet for tablet 81ced6c9edfa4f8491377edfb4e148d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39978
I20260805 23:17:08.995854 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 3335
I20260805 23:17:08.998561 3799 mvcc.cc:205] Tried to move back new op lower bound from 7315340611337969664 to 7315340610925633536. Current Snapshot: MvccSnapshot[applied={T|T < 7315340611337969664}]
I20260805 23:17:09.003599 3806 raft_consensus.cc:2229] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:17:09.004150 3806 raft_consensus.cc:2258] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:17:09.008586 3806 ts_tablet_manager.cc:1905] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:17:09.014168 3806 ts_tablet_manager.cc:1918] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:17:09.023388 3807 tablet_replica.cc:330] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: stopping tablet replica
I20260805 23:17:09.023957 3807 raft_consensus.cc:2229] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:17:09.024490 3807 raft_consensus.cc:2258] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:17:09.027202 3806 log.cc:1196] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/81ced6c9edfa4f8491377edfb4e148d8
I20260805 23:17:09.027431 3807 ts_tablet_manager.cc:1905] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:17:09.027697 3806 ts_tablet_manager.cc:1939] T 81ced6c9edfa4f8491377edfb4e148d8 P 316f3b2b7c0c4808b0b76655d38c33bf: Deleting consensus metadata
I20260805 23:17:09.029116 3233 catalog_manager.cc:4841] TS 316f3b2b7c0c4808b0b76655d38c33bf (127.23.88.67:42867): tablet 81ced6c9edfa4f8491377edfb4e148d8 (table test-table [id=6b6ae9cde3b745c28e00c26fe3b6dc22]) successfully deleted
W20260805 23:17:09.029323 3236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:37793 (error 108)
I20260805 23:17:09.030493 3807 ts_tablet_manager.cc:1918] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:17:09.030586 3807 log.cc:1196] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/81ced6c9edfa4f8491377edfb4e148d8
I20260805 23:17:09.030918 3807 ts_tablet_manager.cc:1939] T 81ced6c9edfa4f8491377edfb4e148d8 P 094469e5b501446fbbb4f56ad5bb5200: Deleting consensus metadata
I20260805 23:17:09.032457 3235 catalog_manager.cc:4841] TS 094469e5b501446fbbb4f56ad5bb5200 (127.23.88.66:45653): tablet 81ced6c9edfa4f8491377edfb4e148d8 (table test-table [id=6b6ae9cde3b745c28e00c26fe3b6dc22]) successfully deleted
I20260805 23:17:09.043296 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 3481
I20260805 23:17:09.045398 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:09.045010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37793 (local address 127.0.0.1:57722)
0805 23:17:09.045206 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:09.045299 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260805 23:17:09.047163 3236 catalog_manager.cc:4584] TS 6cd7278488c547329dfd4d83f23f0a9e (127.23.88.65:37793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81ced6c9edfa4f8491377edfb4e148d8: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37793: connect: Connection refused (error 111)
I20260805 23:17:09.054966 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 3625
I20260805 23:17:09.067066 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 3217
I20260805 23:17:09.075402 23905 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 23:17:09.420688 23905 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 23:17:09.420842 23905 mini_ranger.cc:66] Stopping Ranger...
I20260805 23:17:09.765997 23905 mini_ranger.cc:68] Stopped Ranger
2026-08-05 23:17:09.766 UTC [2782] LOG: received smart shutdown request
2026-08-05 23:17:09.770 UTC [2782] LOG: background worker "logical replication launcher" (PID 2791) exited with exit code 1
2026-08-05 23:17:09.770 UTC [2786] LOG: shutting down
2026-08-05 23:17:09.964 UTC [2782] LOG: database system is shut down
2026-08-05T23:17:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50241 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:17:10 dist-test-slave-vkjp krb5kdc[3818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:17:10 dist-test-slave-vkjp krb5kdc[3818](info): set up 2 sockets
Aug 05 23:17:10 dist-test-slave-vkjp krb5kdc[3818](info): commencing operation
krb5kdc: starting...
W20260805 23:17:11.164898 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:17:11 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:17:11Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangeradmin_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangeradmin_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangerlookup_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangerlookup_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
I20260805 23:17:11.341634 23905 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRmBmWz/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/postgres -l logfile start
2026-08-05 23:17:13.891 UTC [3849] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-05 23:17:13.891 UTC [3849] LOG: listening on IPv4 address "127.23.88.84", port 40025
2026-08-05 23:17:13.895 UTC [3849] LOG: listening on Unix socket "/tmp/.s.PGSQL.40025"
2026-08-05 23:17:13.931 UTC [3852] LOG: database system was shut down at 2026-08-05 23:17:12 UTC
2026-08-05 23:17:13.935 UTC [3849] LOG: database system is ready to accept connections
I20260805 23:17:14.921247 23905 mini_postgres.cc:96] Postgres bound to 40025
2026-08-05 23:17:14.927 UTC [3860] FATAL: database "slave" does not exist
127.23.88.84:40025 - accepting connections
I20260805 23:17:14.928417 23905 mini_ranger.cc:161] Starting Ranger...
I20260805 23:17:14.943472 23905 mini_ranger.cc:84] Created miniranger Postgres user
I20260805 23:17:15.442281 23905 mini_ranger.cc:87] Created ranger Postgres database
I20260805 23:17:15.442360 23905 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin
2026-08-05 23:17:15,730 [I] DB FLAVOR :POSTGRES
2026-08-05 23:17:15,731 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:17:15,731 [I] Checking connection..
2026-08-05 23:17:15,731 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-05 23:17:16,014 [I] Checking connection passed.
2026-08-05 23:17:16,014 [I] --------- Verifying version history table ---------
2026-08-05 23:17:16,014 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:17:16,278 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:17:16,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:17:16,523 [I] Table x_db_version_h does not exist in database ranger
2026-08-05 23:17:16,524 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-05 23:17:16,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-05 23:17:16.797 UTC [3929] WARNING: there is no transaction in progress
2026-08-05 23:17:16,803 [I] create_dbversion_catalog.sql file imported successfully
2026-08-05 23:17:16,803 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-05 23:17:17,046 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-05 23:17:17,046 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:17:17,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-05 23:17:17,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-05 23:17:17,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vkjp.c.gcp-upstream.internal','N') ;"
2026-08-05 23:17:17,837 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:17:18,084 [I] Table x_portal_user does not exist in database ranger
2026-08-05 23:17:18,084 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:17:18,351 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-05 23:17:18,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:17:18,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:17:18,841 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-05 23:17:18,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-05 23:17:19.862 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:19.879 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:19.897 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:19.914 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:19.934 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:19.955 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.025 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.032 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.048 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.059 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.070 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.086 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.096 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.103 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.110 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20.118 UTC [4044] WARNING: there is no transaction in progress
2026-08-05 23:17:20,817 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-05 23:17:20,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-05 23:17:21,090 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-05 23:17:21,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-05 23:17:21,338 [I] Table x_portal_user already exists in database 'ranger'
2026-08-05 23:17:21,338 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-05 23:17:21,581 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-05 23:17:21,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-05 23:17:21,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-05 23:17:22,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vkjp.c.gcp-upstream.internal';"
2026-08-05 23:17:22,373 [I] CORE_DB_SCHEMA import status has been updated
2026-08-05 23:17:22,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-05 23:17:22,637 [I] DB_PATCHES have already been applied
I20260805 23:17:22.643694 23905 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRmBmWz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260805 23:17:22.643782 23905 mini_ranger.cc:193] Using host: 127.23.88.84
I20260805 23:17:22.647419 23905 mini_ranger.cc:227] Ranger admin URL: http://127.23.88.84:54447
Aug 05, 2026 11:17:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:17:22 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:17:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp
Aug 05, 2026 11:17:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 05, 2026 11:17:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 05, 2026 11:17:23 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-admin/ews/webapp].
Aug 05, 2026 11:17:23 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260805 23:17:23.793686 23905 mini_ranger.cc:160] Time spent starting Ranger: real 8.865s user 0.000s sys 0.009s
Aug 05, 2026 11:17:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 05, 2026 11:17:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 05, 2026 11:17:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 05, 2026 11:17:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 05, 2026 11:17:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260805 23:17:46.693665 23905 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.88.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.88.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangerkms_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.88.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/rangerkms_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.88.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.88.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.88.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/HTTP_127.23.88.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 05 23:17:46 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971866, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260805 23:17:46.787127 23905 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260805 23:17:46.797652 23905 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260805 23:17:47.689095 23905 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260805 23:17:47.689183 23905 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms
I20260805 23:17:47.689206 23905 mini_ranger_kms.cc:227] Using postgres at 127.23.88.84:40025
2026-08-05 23:17:47,867 [I] DB FLAVOR :POSTGRES
2026-08-05 23:17:47,868 [I] --------- Verifying Ranger DB connection ---------
2026-08-05 23:17:47,868 [I] Checking connection
2026-08-05 23:17:47,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-05 23:17:48,113 [I] connection success
2026-08-05 23:17:48,114 [I] --------- Verifying Ranger DB tables ---------
2026-08-05 23:17:48,114 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-05 23:17:48,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-05 23:17:48,367 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-05 23:17:48,368 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-05 23:17:48,368 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-05 23:17:48,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.88.84:40025/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-05 23:17:48,629 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260805 23:17:49.150118 23905 mini_ranger_kms.cc:310] Created kms service
I20260805 23:17:49.331946 23905 mini_ranger_kms.cc:326] Created kudu user
I20260805 23:17:49.380064 23905 mini_ranger_kms.cc:343] Created rangerkms user
Aug 05 23:17:49 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971866, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:49.827685 23905 mini_ranger_kms.cc:384] Added ranger policy
I20260805 23:17:49.827863 23905 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRmBmWz/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRmBmWz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260805 23:17:49.827888 23905 mini_ranger_kms.cc:240] Using host: 127.23.88.84
I20260805 23:17:49.832278 23905 mini_ranger_kms.cc:276] Ranger KMS PID: 4233
I20260805 23:17:49.832396 23905 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.88.84:43843
Aug 05, 2026 11:17:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 05, 2026 11:17:50 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 05, 2026 11:17:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 05, 2026 11:17:50 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 05, 2026 11:17:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-kms/ews/webapp].
Aug 05, 2026 11:17:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 05 23:17:52 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:52.180626 23905 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.394s user 0.016s sys 0.145s
I20260805 23:17:52.180763 23905 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260805 23:17:52.180842 23905 mini_ranger_kms.cc:401] 127.23.88.84:43843/kms/v1/keys
Aug 05 23:17:52 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
Aug 05 23:17:52 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971872, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.88.84@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
Aug 05 23:17:55 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:17:55.223269 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41863
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41863
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:55.339139 4286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:55.339438 4286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:55.339481 4286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:55.343652 4286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:17:55.343720 4286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:55.343740 4286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:55.343756 4286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:17:55.343815 4286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:17:55.349618 4286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.88.84:43843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41863
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:55.350785 4286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:55.351859 4286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:17:55.361171 4286 server_base.cc:1034] running on GCE node
W20260805 23:17:55.361148 4296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:55.361129 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:55.361129 4293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:55.361963 4286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:55.363049 4286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:55.364253 4286 hybrid_clock.cc:648] HybridClock initialized: now 1785971875364233 us; error 39 us; skew 500 ppm
Aug 05 23:17:55 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:55.367228 4286 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:17:55.368171 4286 webserver.cc:460] Webserver started at http://127.23.88.126:38909/ using document root <none> and password file <none>
I20260805 23:17:55.368743 4286 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:55.368824 4286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:55.369131 4286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:55 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:55.424674 4286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "dbaaa8b486b943a0bd67e49ddec8a101"
format_stamp: "Formatted at 2026-08-05 23:17:55 on dist-test-slave-vkjp"
server_key: "7ce7eb7cd65b2cd3c49f672cc5ac75ea"
server_key_iv: "05940010d8e8102c7920d1f5ccf47dce"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:55.425413 4286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "dbaaa8b486b943a0bd67e49ddec8a101"
format_stamp: "Formatted at 2026-08-05 23:17:55 on dist-test-slave-vkjp"
server_key: "7ce7eb7cd65b2cd3c49f672cc5ac75ea"
server_key_iv: "05940010d8e8102c7920d1f5ccf47dce"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:55.429903 4286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260805 23:17:55.445773 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:55.447567 4286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:17:55.447731 4286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "dbaaa8b486b943a0bd67e49ddec8a101"
format_stamp: "Formatted at 2026-08-05 23:17:55 on dist-test-slave-vkjp"
server_key: "7ce7eb7cd65b2cd3c49f672cc5ac75ea"
server_key_iv: "05940010d8e8102c7920d1f5ccf47dce"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:55.447911 4286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:55.464660 4286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:55.467935 4286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:55.468150 4286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:55.476595 4286 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41863
I20260805 23:17:55.476665 4359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41863 every 8 connection(s)
I20260805 23:17:55.477650 4286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:17:55.480577 4360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:55.481760 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4286
I20260805 23:17:55.481927 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:17:55.482265 23905 external_mini_cluster.cc:1428] Setting key 56cdc156fc7106f9eeb54d06ef865fc0
I20260805 23:17:55.487726 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Bootstrap starting.
I20260805 23:17:55.490370 4360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:55.491353 4360 log.cc:824] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:55.493541 4360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: No bootstrap required, opened a new log
Aug 05 23:17:55 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:55.497207 4360 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:55.497483 4360 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:55.497560 4360 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbaaa8b486b943a0bd67e49ddec8a101, State: Initialized, Role: FOLLOWER
I20260805 23:17:55.498140 4360 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:55.498317 4360 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:17:55.498430 4360 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:17:55.498549 4360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:55.499768 4360 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:55.500257 4360 leader_election.cc:302] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbaaa8b486b943a0bd67e49ddec8a101; no voters:
I20260805 23:17:55.500656 4360 leader_election.cc:288] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:17:55.500736 4365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:17:55.501031 4365 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 LEADER]: Becoming Leader. State: Replica: dbaaa8b486b943a0bd67e49ddec8a101, State: Running, Role: LEADER
I20260805 23:17:55.501402 4365 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:55.501478 4363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:55.483929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54358 (local address 127.23.88.126:41863)
0805 23:17:55.484435 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:55.484455 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:55.484500 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:17:55.485347 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:55.485371 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:55.485814 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:55.486181 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:55.487217 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:55.488658 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:55.489994 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:55.490748 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:55.490913 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:55.495047 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:55.495072 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:55.495122 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:55.495161 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:55.497485 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:55.498126 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:55.498131 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:55.498137 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:55.498284 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:55.498681 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:55.498685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:55.498687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:55.499303 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:55.499539 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:55.499873 (+ 334us) server_negotiation.cc:299] Negotiation successful
0805 23:17:55.500188 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":155,"threads_started":1}
I20260805 23:17:55.502130 4360 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:17:55.502699 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbaaa8b486b943a0bd67e49ddec8a101. Latest consensus state: current_term: 1 leader_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } } }
I20260805 23:17:55.502908 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:55.503218 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } } }
I20260805 23:17:55.503391 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:55.504683 4360 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:17:55.505300 4371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:17:55.515448 4371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:17:55.524497 4371 catalog_manager.cc:1347] Generated new cluster ID: b9441474fc024103a5738947f700d8b0
I20260805 23:17:55.524586 4371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:17:55.554667 4371 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:17:55.557471 4371 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:17:55.564535 4371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Generated new TSK 0
I20260805 23:17:55.566602 4371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:17:56.366218 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:56.487706 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:56.488116 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:56.488224 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:56.493134 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:56.493219 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:56.493307 4408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:17:56.498950 4408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:56.500146 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:56.501186 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:56.513242 4414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:56.513221 4416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:56.513247 4413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:56.514581 4408 server_base.cc:1034] running on GCE node
I20260805 23:17:56.515149 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:56.515957 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:56.517112 4408 hybrid_clock.cc:648] HybridClock initialized: now 1785971876517045 us; error 82 us; skew 500 ppm
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:56.526453 4408 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:17:56.527802 4408 webserver.cc:460] Webserver started at http://127.23.88.65:33149/ using document root <none> and password file <none>
I20260805 23:17:56.528580 4408 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:56.528682 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:56.529011 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:56.546833 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "0f38cbea1ace4f029d63bc3eac5c781d"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "6f86731ede6762d8b989197b7b5d9e31"
server_key_iv: "5100f8ee4bada3a048350dc9f438adff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.551119 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "0f38cbea1ace4f029d63bc3eac5c781d"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "6f86731ede6762d8b989197b7b5d9e31"
server_key_iv: "5100f8ee4bada3a048350dc9f438adff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.557331 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260805 23:17:56.572999 4425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.574426 4408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:17:56.574571 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "0f38cbea1ace4f029d63bc3eac5c781d"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "6f86731ede6762d8b989197b7b5d9e31"
server_key_iv: "5100f8ee4bada3a048350dc9f438adff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.574718 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:56.591133 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:56.594937 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:56.595228 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:56.596045 4408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:56.597141 4408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:56.597250 4408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.597389 4408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:56.597483 4408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.610354 4408 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:40729
I20260805 23:17:56.611677 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:17:56.612627 4543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:40729 every 8 connection(s)
I20260805 23:17:56.614418 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4408
I20260805 23:17:56.614570 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:17:56.614939 23905 external_mini_cluster.cc:1428] Setting key 45ac5934f44d48f293a333515177b41b
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:17:56.634770 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:56.615472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.65:51143)
0805 23:17:56.619459 (+ 3987us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:56.619523 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:56.620780 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:56.621648 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:56.621655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:56.621937 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:56.622427 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:56.622441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.623342 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.623349 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:56.624780 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:56.624795 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.625681 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.625689 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:56.625935 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:56.626693 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:56.626710 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:56.628606 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:56.631230 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:56.631236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:56.631255 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:56.631627 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:56.632000 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:56.632005 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:56.632007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:56.632202 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:56.632715 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:56.632722 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:56.633083 (+ 361us) client_negotiation.cc:769] Sending connection context
0805 23:17:56.633307 (+ 224us) client_negotiation.cc:240] Negotiation successful
0805 23:17:56.633664 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3721,"thread_start_us":143,"threads_started":1}
I20260805 23:17:56.635555 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:56.615689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:51143 (local address 127.23.88.126:41863)
0805 23:17:56.619861 (+ 4172us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:56.619867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:56.619886 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:17:56.621208 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:56.621213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:56.621278 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:56.621375 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:56.622584 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.623214 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.624950 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.625355 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.625474 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:56.628851 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:56.628857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:56.628862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:56.628894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:56.631055 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:56.631763 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:56.631767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:56.631769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:56.631881 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:56.632324 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:56.632327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:56.632329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:56.632562 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:56.632659 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:56.635242 (+ 2583us) server_negotiation.cc:299] Negotiation successful
0805 23:17:56.635385 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4080,"thread_start_us":91,"threads_started":1}
I20260805 23:17:56.637455 4545 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:56.637703 4545 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:56.638299 4545 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:56.640215 4321 ts_manager.cc:194] Registered new tserver with Master: 0f38cbea1ace4f029d63bc3eac5c781d (127.23.88.65:40729)
I20260805 23:17:56.642019 4321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:51143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:17:56.667691 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260805 23:17:56.838766 4553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:56.839203 4553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:56.839347 4553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:56.846405 4553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:56.846498 4553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:56.846597 4553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:17:56.855391 4553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:56.856833 4553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:56.858304 4553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:56.871690 4559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:56.871857 4561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:56.871690 4563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:56.872123 4553 server_base.cc:1034] running on GCE node
I20260805 23:17:56.872977 4553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:56.873868 4553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:56.875159 4553 hybrid_clock.cc:648] HybridClock initialized: now 1785971876875130 us; error 44 us; skew 500 ppm
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:56.879293 4553 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:17:56.880971 4553 webserver.cc:460] Webserver started at http://127.23.88.66:39365/ using document root <none> and password file <none>
I20260805 23:17:56.881841 4553 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:56.881953 4553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:56.882279 4553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:56.906054 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "2aef29f0c54a4b1e8b1b58937a726d26"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "d08da2b82e5b2ae0b72cda6dffbf7908"
server_key_iv: "4c0c1d83a11d2b5a0da0be8edf02a6c5"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.907019 4553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "2aef29f0c54a4b1e8b1b58937a726d26"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "d08da2b82e5b2ae0b72cda6dffbf7908"
server_key_iv: "4c0c1d83a11d2b5a0da0be8edf02a6c5"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.912895 4553 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260805 23:17:56.933501 4570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.935415 4553 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260805 23:17:56.935559 4553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "2aef29f0c54a4b1e8b1b58937a726d26"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "d08da2b82e5b2ae0b72cda6dffbf7908"
server_key_iv: "4c0c1d83a11d2b5a0da0be8edf02a6c5"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:56.935714 4553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:56.949008 4553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:56.952414 4553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:56.952641 4553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:56.953467 4553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:56.954555 4553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:56.954632 4553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.954733 4553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:56.954792 4553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:56.965776 4553 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:40337
I20260805 23:17:56.967208 4553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:17:56.967849 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4553
I20260805 23:17:56.968008 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:17:56.968078 4689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:40337 every 8 connection(s)
I20260805 23:17:56.968389 23905 external_mini_cluster.cc:1428] Setting key faa78892047100ca9d06f047d5955322
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Aug 05 23:17:56 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:17:56.992682 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:56.973914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:52631 (local address 127.23.88.126:41863)
0805 23:17:56.974205 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:56.974212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:56.974743 (+ 531us) server_negotiation.cc:407] Connection header received
0805 23:17:56.976405 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:56.976410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:56.976471 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:56.976541 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:56.978813 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.979753 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.981713 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.982154 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.982230 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:56.987280 (+ 5050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:56.987288 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:56.987294 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:56.987336 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:56.989428 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:56.990239 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:56.990244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:56.990248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:56.990358 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:56.990907 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:56.990914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:56.990916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:56.991244 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:56.991336 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:56.992229 (+ 893us) server_negotiation.cc:299] Negotiation successful
0805 23:17:56.992402 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260805 23:17:56.994544 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:56.972692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.66:52631)
0805 23:17:56.974567 (+ 1875us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:56.974631 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:56.976104 (+ 1473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:56.977050 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:56.977063 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:56.977496 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:56.978574 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:56.978602 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.979991 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.979998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:56.981538 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:56.981553 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:56.982429 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:56.982432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:56.982588 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:56.983349 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:56.983368 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:56.986986 (+ 3618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:56.989617 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:56.989626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:56.989649 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:56.990092 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:56.990507 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:56.990512 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:56.990515 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:56.990761 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:56.991440 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:56.991451 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:56.991868 (+ 417us) client_negotiation.cc:769] Sending connection context
0805 23:17:56.992914 (+ 1046us) client_negotiation.cc:240] Negotiation successful
0805 23:17:56.993319 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1591,"thread_start_us":151,"threads_started":1}
I20260805 23:17:56.996145 4690 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:56.996417 4690 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:56.997037 4690 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:56.998346 4321 ts_manager.cc:194] Registered new tserver with Master: 2aef29f0c54a4b1e8b1b58937a726d26 (127.23.88.66:40337)
I20260805 23:17:56.999197 4321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:52631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:17:57.015300 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:57.165935 4695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:57.166421 4695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:57.166510 4695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:57.172631 4695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:57.172712 4695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:57.172799 4695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:17:57.180740 4695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:57.182186 4695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:57.183562 4695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:57.195695 4701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:57.195699 4700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:57.195809 4703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:57.196949 4695 server_base.cc:1034] running on GCE node
I20260805 23:17:57.197396 4695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:57.198153 4695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:57.199314 4695 hybrid_clock.cc:648] HybridClock initialized: now 1785971877199291 us; error 34 us; skew 500 ppm
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:57.203743 4695 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:17:57.204919 4695 webserver.cc:460] Webserver started at http://127.23.88.67:39527/ using document root <none> and password file <none>
I20260805 23:17:57.205667 4695 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:57.205808 4695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:17:57.206086 4695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:57.221799 4695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "0688058f651b4466ac62dee86aca61ab"
format_stamp: "Formatted at 2026-08-05 23:17:57 on dist-test-slave-vkjp"
server_key: "0be3d7bda44e6115cb9ae6485044604d"
server_key_iv: "33f4db56737d880b1fca49ecb899e909"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:57.222574 4695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "0688058f651b4466ac62dee86aca61ab"
format_stamp: "Formatted at 2026-08-05 23:17:57 on dist-test-slave-vkjp"
server_key: "0be3d7bda44e6115cb9ae6485044604d"
server_key_iv: "33f4db56737d880b1fca49ecb899e909"
server_key_version: "kuduclusterkey@0"
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971876, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:57.228170 4695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260805 23:17:57.261353 4712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.263165 4695 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260805 23:17:57.263388 4695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "0688058f651b4466ac62dee86aca61ab"
format_stamp: "Formatted at 2026-08-05 23:17:57 on dist-test-slave-vkjp"
server_key: "0be3d7bda44e6115cb9ae6485044604d"
server_key_iv: "33f4db56737d880b1fca49ecb899e909"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:57.263614 4695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:57.280792 4695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:57.284344 4695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:57.284595 4695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:57.285404 4695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:57.286448 4695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:57.286554 4695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.286650 4695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:57.286715 4695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.300977 4695 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:37365
I20260805 23:17:57.302294 4695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:17:57.304051 4830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:37365 every 8 connection(s)
I20260805 23:17:57.310536 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4695
I20260805 23:17:57.310679 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:17:57.311062 23905 external_mini_cluster.cc:1428] Setting key 21c9fd978e644b3fe1b0cc627a6e4a67
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:57.325062 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:57.307936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:43885 (local address 127.23.88.126:41863)
0805 23:17:57.308077 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:57.308082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:57.308586 (+ 504us) server_negotiation.cc:407] Connection header received
0805 23:17:57.310312 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:57.310316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:57.310369 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:57.310420 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:57.312532 (+ 2112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.313367 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.315204 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.315561 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.315634 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:57.319640 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:57.319645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:57.319648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:57.319677 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:57.322038 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:57.322826 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:57.322831 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:57.322833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:57.322940 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:57.323473 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:57.323477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:57.323480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:57.323757 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:57.323865 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:57.324671 (+ 806us) server_negotiation.cc:299] Negotiation successful
0805 23:17:57.324844 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:17:57.327322 4833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:57.306733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.67:43885)
0805 23:17:57.308426 (+ 1693us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:57.308482 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:57.310055 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:57.310937 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:57.310946 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:57.311549 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:57.312358 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:57.312376 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.313525 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.313528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:57.315030 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:57.315041 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.315831 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.315834 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:57.316089 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:57.317111 (+ 1022us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:57.317132 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:57.319434 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:57.322223 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:57.322230 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:57.322260 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:57.322673 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:57.323074 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:57.323099 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:57.323101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:57.323331 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:57.323951 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:57.323958 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:57.324334 (+ 376us) client_negotiation.cc:769] Sending connection context
0805 23:17:57.325631 (+ 1297us) client_negotiation.cc:240] Negotiation successful
0805 23:17:57.326030 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1441,"thread_start_us":227,"threads_started":1}
I20260805 23:17:57.329705 4831 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:57.329977 4831 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:57.330570 4831 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:57.331782 4321 ts_manager.cc:194] Registered new tserver with Master: 0688058f651b4466ac62dee86aca61ab (127.23.88.67:37365)
I20260805 23:17:57.332520 4321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:43885
I20260805 23:17:57.332995 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:17:57.333835 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4408
I20260805 23:17:57.349810 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4553
I20260805 23:17:57.358397 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4695
I20260805 23:17:57.366364 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4286
I20260805 23:17:57.373673 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41863
--webserver_interface=127.23.88.126
--webserver_port=38909
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41863
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:17:57.486624 4834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:57.486904 4834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:57.487006 4834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:57.491381 4834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:17:57.491437 4834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:57.491457 4834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:57.491473 4834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:17:57.491509 4834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:17:57.497035 4834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41863
--ranger_config_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=38909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:57.498124 4834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:57.499183 4834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:57.507865 4840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:57.507905 4834 server_base.cc:1034] running on GCE node
W20260805 23:17:57.507900 4841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:57.507838 4843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:57.508718 4834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:57.509898 4834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:57.511230 4834 hybrid_clock.cc:648] HybridClock initialized: now 1785971877511212 us; error 31 us; skew 500 ppm
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:57.514108 4834 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:17:57.515177 4834 webserver.cc:460] Webserver started at http://127.23.88.126:38909/ using document root <none> and password file <none>
I20260805 23:17:57.515712 4834 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:57.515822 4834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:57.559294 4834 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:17:57.562032 4851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.563668 4834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:17:57.563867 4834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "dbaaa8b486b943a0bd67e49ddec8a101"
format_stamp: "Formatted at 2026-08-05 23:17:55 on dist-test-slave-vkjp"
server_key: "7ce7eb7cd65b2cd3c49f672cc5ac75ea"
server_key_iv: "05940010d8e8102c7920d1f5ccf47dce"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:57.564535 4834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:57.600399 4834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:57.603504 4834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:57.603721 4834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:57.611122 4834 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41863
I20260805 23:17:57.611224 4905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41863 every 8 connection(s)
I20260805 23:17:57.612275 4834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:17:57.616721 4907 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:17:57.618397 4907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Bootstrap starting.
I20260805 23:17:57.622283 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4834
I20260805 23:17:57.622822 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:40729
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:41863
--webserver_port=33149
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:17:57.636174 4907 log.cc:824] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:57.643378 4907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:17:57.643816 4907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Bootstrap complete.
I20260805 23:17:57.648237 4907 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:57.648648 4907 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbaaa8b486b943a0bd67e49ddec8a101, State: Initialized, Role: FOLLOWER
I20260805 23:17:57.649161 4907 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:57.649276 4907 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:17:57.649374 4907 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:17:57.649473 4907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:17:57.651540 4907 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:57.651886 4907 leader_election.cc:302] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbaaa8b486b943a0bd67e49ddec8a101; no voters:
I20260805 23:17:57.652254 4907 leader_election.cc:288] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:17:57.652351 4911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:17:57.652586 4911 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [term 2 LEADER]: Becoming Leader. State: Replica: dbaaa8b486b943a0bd67e49ddec8a101, State: Running, Role: LEADER
I20260805 23:17:57.652961 4911 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } }
I20260805 23:17:57.653596 4907 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:17:57.654315 4912 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } } }
I20260805 23:17:57.654446 4912 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:57.655179 4913 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbaaa8b486b943a0bd67e49ddec8a101. Latest consensus state: current_term: 2 leader_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbaaa8b486b943a0bd67e49ddec8a101" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41863 } } }
I20260805 23:17:57.655275 4913 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101 [sys.catalog]: This master's current role is: LEADER
I20260805 23:17:57.655704 4918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:17:57.656121 4907 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260805 23:17:57.662640 4918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:17:57.663614 4918 catalog_manager.cc:1259] Loaded cluster ID: b9441474fc024103a5738947f700d8b0
I20260805 23:17:57.663748 4918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:17:57.666499 4918 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:17:57.667398 4918 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Loaded TSK: 0
I20260805 23:17:57.669230 4918 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:17:57.751684 4909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:57.752023 4909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:57.752131 4909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:57.757021 4909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:57.757124 4909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:57.757234 4909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:17:57.763008 4909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:40729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=33149
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:57.764439 4909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:57.765893 4909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:57.778244 4948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:57.778244 4950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:57.778751 4947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:57.779448 4909 server_base.cc:1034] running on GCE node
I20260805 23:17:57.779949 4909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:57.780673 4909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:57.781884 4909 hybrid_clock.cc:648] HybridClock initialized: now 1785971877781827 us; error 44 us; skew 500 ppm
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:57.784963 4909 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:17:57.786013 4909 webserver.cc:460] Webserver started at http://127.23.88.65:33149/ using document root <none> and password file <none>
I20260805 23:17:57.786662 4909 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:57.786765 4909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:57.814110 4909 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:17:57.821611 4958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.823397 4909 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260805 23:17:57.823558 4909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "0f38cbea1ace4f029d63bc3eac5c781d"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "6f86731ede6762d8b989197b7b5d9e31"
server_key_iv: "5100f8ee4bada3a048350dc9f438adff"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:57.824327 4909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:57.867586 4909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:57.871762 4909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:57.872002 4909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:57.872805 4909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:57.873939 4909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:57.874012 4909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.874075 4909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:57.874104 4909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:57.885262 4909 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:40729
I20260805 23:17:57.886617 4909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:17:57.892004 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 4909
I20260805 23:17:57.892515 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:40337
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:41863
--webserver_port=39365
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:17:57.895771 5080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:40729 every 8 connection(s)
Aug 05 23:17:57 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971877, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:57.912573 5084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:57.893612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.65:47583)
0805 23:17:57.894365 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:57.894428 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:57.895438 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:57.897052 (+ 1614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:57.897063 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:57.897393 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:57.898057 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:57.898078 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.900442 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.900446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:57.901838 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:57.901851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.902930 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.902934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:57.903157 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:57.903998 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:57.904022 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:57.905962 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:57.908789 (+ 2827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:57.908796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:57.908812 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:57.909184 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:57.909671 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:57.909675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:57.909677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:57.909872 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:57.910665 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:57.910672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:57.910975 (+ 303us) client_negotiation.cc:769] Sending connection context
0805 23:17:57.911190 (+ 215us) client_negotiation.cc:240] Negotiation successful
0805 23:17:57.911537 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":173,"threads_started":1}
I20260805 23:17:57.914723 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:57.895197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:47583 (local address 127.23.88.126:41863)
0805 23:17:57.895722 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:57.895745 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:57.895790 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:17:57.896131 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:57.896148 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:57.896568 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:57.898370 (+ 1802us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:57.898690 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.900263 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.902001 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:57.902736 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:57.905116 (+ 2380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:57.906208 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:57.906216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:57.906233 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:57.906268 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:57.908639 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:57.909348 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:57.909354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:57.909359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:57.909519 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:57.910001 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:57.910004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:57.910006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:57.910522 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:57.913289 (+ 2767us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:57.913501 (+ 212us) server_negotiation.cc:299] Negotiation successful
0805 23:17:57.913799 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":153,"threads_started":1}
I20260805 23:17:57.917264 5081 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:57.917529 5081 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:57.918104 5081 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:57.920344 5081 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:41863 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 23:17:57.920488 5081 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:57.920668 5081 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:57.921464 5081 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:57.921649 5081 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:57.922353 5081 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:17:58.040642 5083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:58.040934 5083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:58.040980 5083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:58.045635 5083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:58.045732 5083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:58.045852 5083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:17:58.051873 5083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:40337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=39365
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:58.053323 5083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:58.054754 5083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:58.065724 5093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:58.065716 5090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:58.065716 5091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:58.066556 5083 server_base.cc:1034] running on GCE node
I20260805 23:17:58.067123 5083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:58.067946 5083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:58.069144 5083 hybrid_clock.cc:648] HybridClock initialized: now 1785971878069106 us; error 59 us; skew 500 ppm
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:58.072451 5083 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:17:58.073676 5083 webserver.cc:460] Webserver started at http://127.23.88.66:39365/ using document root <none> and password file <none>
I20260805 23:17:58.074462 5083 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:58.074564 5083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:58.103820 5083 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260805 23:17:58.107028 5104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.108587 5083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:17:58.108719 5083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "2aef29f0c54a4b1e8b1b58937a726d26"
format_stamp: "Formatted at 2026-08-05 23:17:56 on dist-test-slave-vkjp"
server_key: "d08da2b82e5b2ae0b72cda6dffbf7908"
server_key_iv: "4c0c1d83a11d2b5a0da0be8edf02a6c5"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:58.109444 5083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:58.124054 5083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:58.127746 5083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:58.127997 5083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:58.128715 5083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:58.129726 5083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:58.129794 5083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.129859 5083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:58.129885 5083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.144548 5083 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:40337
I20260805 23:17:58.144608 5224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:40337 every 8 connection(s)
I20260805 23:17:58.146034 5083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:17:58.153730 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5083
I20260805 23:17:58.154296 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.88.84:43843/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:37365
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:41863
--webserver_port=39527
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:42107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:58.166123 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:58.148751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.66:44053)
0805 23:17:58.149292 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:58.149340 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:58.150304 (+ 964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:58.150944 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:58.150954 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:58.151372 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:58.152080 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:58.152102 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.153040 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.153043 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:58.155040 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:58.155059 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.156407 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.156412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:58.156643 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:58.157509 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:58.157533 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:58.159480 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:58.161994 (+ 2514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:58.162002 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:58.162021 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:58.162418 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:58.162777 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:58.162783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:58.162788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:58.163024 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:58.163707 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:58.163717 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:58.164145 (+ 428us) client_negotiation.cc:769] Sending connection context
0805 23:17:58.164390 (+ 245us) client_negotiation.cc:240] Negotiation successful
0805 23:17:58.164828 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":136,"threads_started":1}
I20260805 23:17:58.167261 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:58.148365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:44053 (local address 127.23.88.126:41863)
0805 23:17:58.148524 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:58.148528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:58.149481 (+ 953us) server_negotiation.cc:407] Connection header received
0805 23:17:58.150537 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:58.150552 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:58.150626 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:58.150745 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:58.152329 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.152885 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.155267 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.155747 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.155869 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:58.159714 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:58.159719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:58.159726 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:58.159770 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:58.161852 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.162559 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.162563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.162565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.162660 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.163221 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.163228 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.163231 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.163527 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:58.163640 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:58.166799 (+ 3159us) server_negotiation.cc:299] Negotiation successful
0805 23:17:58.167003 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:17:58.168756 5225 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:58.169147 5225 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.169826 5225 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:58.171864 5225 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:41863 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 23:17:58.171996 5225 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.172183 5225 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:58.172966 5225 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.173159 5225 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:58.173862 5225 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:17:58.300796 5228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:17:58.301244 5228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:17:58.301339 5228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:17:58.307947 5228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:17:58.308044 5228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:17:58.308135 5228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:17:58.316464 5228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:42107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.88.84:43843/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:37365
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=39527
--tserver_master_addrs=127.23.88.126:41863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:17:58.317939 5228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:17:58.319398 5228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:17:58.332086 5238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:58.332095 5235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:17:58.333249 5236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:17:58.333524 5228 server_base.cc:1034] running on GCE node
I20260805 23:17:58.334100 5228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:17:58.335035 5228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:17:58.336261 5228 hybrid_clock.cc:648] HybridClock initialized: now 1785971878336227 us; error 55 us; skew 500 ppm
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:58.339532 5228 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:17:58.340543 5228 webserver.cc:460] Webserver started at http://127.23.88.67:39527/ using document root <none> and password file <none>
I20260805 23:17:58.341179 5228 fs_manager.cc:359] Metadata directory not provided
I20260805 23:17:58.341301 5228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:58.363173 5228 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:17:58.366405 5247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.368089 5228 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:17:58.368230 5228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "0688058f651b4466ac62dee86aca61ab"
format_stamp: "Formatted at 2026-08-05 23:17:57 on dist-test-slave-vkjp"
server_key: "0be3d7bda44e6115cb9ae6485044604d"
server_key_iv: "33f4db56737d880b1fca49ecb899e909"
server_key_version: "kuduclusterkey@0"
I20260805 23:17:58.369000 5228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:17:58.394745 5228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:17:58.398772 5228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:17:58.398991 5228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:17:58.399796 5228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:17:58.400959 5228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:17:58.401047 5228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.401105 5228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:17:58.401136 5228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:17:58.413349 5228 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:37365
I20260805 23:17:58.414726 5228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:17:58.415264 5367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:37365 every 8 connection(s)
I20260805 23:17:58.417551 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5228
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:17:58.432071 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:58.418260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54362 (local address 127.23.88.126:41863)
0805 23:17:58.419749 (+ 1489us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:58.419754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:58.419779 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:17:58.419818 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:58.419821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:58.419866 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:58.419953 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:58.422939 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.423543 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.424741 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.425080 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.425195 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:58.426735 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:58.426739 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:58.426741 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:58.426775 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:58.428883 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.429316 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.429321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.429323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.429433 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.429731 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.429737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.429742 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.429974 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:58.430176 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:58.431730 (+ 1554us) server_negotiation.cc:299] Negotiation successful
0805 23:17:58.431905 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":86,"threads_started":1}
I20260805 23:17:58.432723 5085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:58.417045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:46731 (local address 127.23.88.126:41863)
0805 23:17:58.417200 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:58.417205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:58.418105 (+ 900us) server_negotiation.cc:407] Connection header received
0805 23:17:58.419549 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:58.419553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:58.419616 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:58.419711 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:58.421399 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.422227 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.423934 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.424239 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.424303 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:58.427586 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:58.427590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:58.427593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:58.427643 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:58.429913 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.430499 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.430505 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.430510 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.430595 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:58.431178 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:58.431185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:58.431186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:58.431361 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:58.431435 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:58.432435 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0805 23:17:58.432565 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:17:58.434159 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:58.417452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41863 (local address 127.23.88.67:46731)
0805 23:17:58.417941 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:58.418000 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:58.419323 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:58.420032 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:58.420043 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:58.420482 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:17:58.421201 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:58.421225 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.422415 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.422418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:58.423786 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:58.423798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:58.424487 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:58.424490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:58.424628 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:58.425587 (+ 959us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:17:58.425604 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:17:58.427410 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:17:58.430037 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:58.430043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:58.430061 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:58.430386 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:58.430712 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:17:58.430716 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:17:58.430720 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:17:58.430915 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:17:58.431496 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:17:58.431501 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:17:58.431846 (+ 345us) client_negotiation.cc:769] Sending connection context
0805 23:17:58.432072 (+ 226us) client_negotiation.cc:240] Negotiation successful
0805 23:17:58.432436 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":149,"threads_started":1}
I20260805 23:17:58.435670 5369 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41863
I20260805 23:17:58.435916 5369 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.436517 5369 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:58.438292 5369 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:41863 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260805 23:17:58.438411 5369 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.438575 5369 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:58.439448 5369 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.439616 5369 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
W20260805 23:17:58.440395 5369 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 05 23:17:58 dist-test-slave-vkjp krb5kdc[3818](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:17:58.922600 5081 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:58.922961 5081 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:58.924136 4869 ts_manager.cc:194] Registered new tserver with Master: 0f38cbea1ace4f029d63bc3eac5c781d (127.23.88.65:40729)
I20260805 23:17:58.925398 4869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:47583
I20260805 23:17:59.174098 5225 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:59.174454 5225 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:59.175326 4869 ts_manager.cc:194] Registered new tserver with Master: 2aef29f0c54a4b1e8b1b58937a726d26 (127.23.88.66:40337)
I20260805 23:17:59.176126 4869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:44053
Aug 05 23:17:59 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:59.440636 5369 heartbeater.cc:464] Registering TS with master...
I20260805 23:17:59.440968 5369 heartbeater.cc:511] Master 127.23.88.126:41863 requested a full tablet report, sending...
I20260805 23:17:59.442042 4869 ts_manager.cc:194] Registered new tserver with Master: 0688058f651b4466ac62dee86aca61ab (127.23.88.67:37365)
I20260805 23:17:59.443145 4869 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:46731
I20260805 23:17:59.450083 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:17:59.463797 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.452107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54370 (local address 127.23.88.126:41863)
0805 23:17:59.452357 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.452362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.452639 (+ 277us) server_negotiation.cc:407] Connection header received
0805 23:17:59.452773 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.452776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.452830 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.452920 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:17:59.453793 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.454655 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.455976 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.456314 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.456412 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.458253 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:17:59.458257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:17:59.458270 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:17:59.458313 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:17:59.461292 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:59.461845 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:59.461849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:59.461850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:59.461949 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:17:59.462283 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:17:59.462286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:17:59.462288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:17:59.462653 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:17:59.462883 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.463181 (+ 298us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.463473 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":84,"threads_started":1}
I20260805 23:17:59.469321 4869 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:17:59.474123 4869 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:17:59.499789 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.487133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:40337 (local address 127.0.0.1:56512)
0805 23:17:59.488341 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.488400 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:59.488578 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:59.493599 (+ 5021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:59.493607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:59.493633 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:59.493801 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.493810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.495521 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.495525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.496966 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.496978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.499152 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.499155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.499281 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.499390 (+ 109us) client_negotiation.cc:769] Sending connection context
0805 23:17:59.499500 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 23:17:59.499625 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":1,"thread_start_us":245,"threads_started":1}
I20260805 23:17:59.500236 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.488405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.23.88.66:40337)
0805 23:17:59.492723 (+ 4318us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.492733 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.492765 (+ 32us) server_negotiation.cc:407] Connection header received
0805 23:17:59.492855 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.492862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.493202 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.493395 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:59.494019 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.495378 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.497120 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.498800 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.498902 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.499048 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.500003 (+ 955us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.500109 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2979,"thread_start_us":203,"threads_started":1}
I20260805 23:17:59.504241 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.490650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:40729 (local address 127.0.0.1:40058)
0805 23:17:59.495938 (+ 5288us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.495956 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:59.496127 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:59.497109 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:59.497113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:59.497133 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:59.497230 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.497236 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.498897 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.498901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.502131 (+ 3230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.502145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.503762 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.503766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.503874 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.503900 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:17:59.503984 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 23:17:59.504063 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5213,"mutex_wait_us":764,"thread_start_us":194,"threads_started":1}
I20260805 23:17:59.503929 5141 tablet_service.cc:1467] Processing CreateTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 (DEFAULT_TABLE table=test-table [id=37431c9b9e2946fa802ff9044a07c5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:59.505043 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.490781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.23.88.65:40729)
0805 23:17:59.495756 (+ 4975us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.495765 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.496264 (+ 499us) server_negotiation.cc:407] Connection header received
0805 23:17:59.496361 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.496367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.496682 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.496855 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:59.497393 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.498735 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.502418 (+ 3683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.503403 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.503528 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.503685 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.504673 (+ 988us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.504885 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4829,"thread_start_us":94,"threads_started":1}
I20260805 23:17:59.505134 5141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b41f332784481ea2053f7f86fcf4c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:59.508780 5011 tablet_service.cc:1467] Processing CreateTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 (DEFAULT_TABLE table=test-table [id=37431c9b9e2946fa802ff9044a07c5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:59.510003 5011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b41f332784481ea2053f7f86fcf4c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:59.511854 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.490320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:37365 (local address 127.0.0.1:50974)
0805 23:17:59.492428 (+ 2108us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.492445 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:59.492556 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:59.506868 (+ 14312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:59.506872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:59.506898 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:59.507011 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.507019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.508775 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.508779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.510052 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.510064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.511385 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.511389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.511500 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.511533 (+ 33us) client_negotiation.cc:769] Sending connection context
0805 23:17:59.511590 (+ 57us) client_negotiation.cc:240] Negotiation successful
0805 23:17:59.511691 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2024,"thread_start_us":110,"threads_started":1}
I20260805 23:17:59.512657 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.490784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50974 (local address 127.23.88.67:37365)
0805 23:17:59.499543 (+ 8759us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.499552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.499582 (+ 30us) server_negotiation.cc:407] Connection header received
0805 23:17:59.499669 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.499675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.506687 (+ 7012us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.507216 (+ 529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:59.507385 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.508623 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.510231 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.511216 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.512170 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.512330 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.512391 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.512508 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8626,"thread_start_us":134,"threads_started":1}
I20260805 23:17:59.515704 5283 tablet_service.cc:1467] Processing CreateTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 (DEFAULT_TABLE table=test-table [id=37431c9b9e2946fa802ff9044a07c5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:17:59.516801 5405 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Bootstrap starting.
I20260805 23:17:59.516860 5283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35b41f332784481ea2053f7f86fcf4c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:17:59.521071 5405 tablet_bootstrap.cc:654] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:59.522038 5405 log.cc:824] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:59.523533 5406 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: Bootstrap starting.
I20260805 23:17:59.527413 5406 tablet_bootstrap.cc:654] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:59.528003 5408 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Bootstrap starting.
I20260805 23:17:59.530676 5408 tablet_bootstrap.cc:654] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Neither blocks nor log segments found. Creating new log.
I20260805 23:17:59.532078 5408 log.cc:824] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:59.532151 5406 log.cc:824] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:17:59.536661 5408 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: No bootstrap required, opened a new log
I20260805 23:17:59.536901 5408 ts_tablet_manager.cc:1397] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260805 23:17:59.538493 5405 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: No bootstrap required, opened a new log
I20260805 23:17:59.538707 5405 ts_tablet_manager.cc:1397] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Time spent bootstrapping tablet: real 0.022s user 0.006s sys 0.000s
I20260805 23:17:59.541900 5408 raft_consensus.cc:348] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.542219 5408 raft_consensus.cc:374] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:59.542275 5408 raft_consensus.cc:729] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0688058f651b4466ac62dee86aca61ab, State: Initialized, Role: FOLLOWER
I20260805 23:17:59.543067 5408 consensus_queue.cc:260] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.544632 5408 ts_tablet_manager.cc:1428] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260805 23:17:59.546882 5406 tablet_bootstrap.cc:492] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: No bootstrap required, opened a new log
I20260805 23:17:59.547214 5406 ts_tablet_manager.cc:1397] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20260805 23:17:59.547370 5369 heartbeater.cc:503] Master 127.23.88.126:41863 was elected leader, sending a full tablet report...
I20260805 23:17:59.550060 5405 raft_consensus.cc:348] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.550519 5405 raft_consensus.cc:374] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:59.550611 5405 raft_consensus.cc:729] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aef29f0c54a4b1e8b1b58937a726d26, State: Initialized, Role: FOLLOWER
I20260805 23:17:59.551283 5405 consensus_queue.cc:260] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.554394 5406 raft_consensus.cc:348] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.554776 5406 raft_consensus.cc:374] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:17:59.554881 5406 raft_consensus.cc:729] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f38cbea1ace4f029d63bc3eac5c781d, State: Initialized, Role: FOLLOWER
I20260805 23:17:59.555519 5406 consensus_queue.cc:260] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.556813 5406 ts_tablet_manager.cc:1428] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260805 23:17:59.557248 5081 heartbeater.cc:503] Master 127.23.88.126:41863 was elected leader, sending a full tablet report...
I20260805 23:17:59.557657 5405 ts_tablet_manager.cc:1428] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Time spent starting tablet: real 0.019s user 0.009s sys 0.000s
I20260805 23:17:59.558068 5225 heartbeater.cc:503] Master 127.23.88.126:41863 was elected leader, sending a full tablet report...
I20260805 23:17:59.563752 5412 raft_consensus.cc:482] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:17:59.564015 5412 raft_consensus.cc:504] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.565402 5412 leader_election.cc:288] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0688058f651b4466ac62dee86aca61ab (127.23.88.67:37365), 0f38cbea1ace4f029d63bc3eac5c781d (127.23.88.65:40729)
I20260805 23:17:59.571534 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.565295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:37365 (local address 127.23.88.66:49175)
0805 23:17:59.566163 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.566178 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:59.566400 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:59.566762 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:59.566765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:59.566780 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:59.567068 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.567075 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.568740 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.568743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.570119 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.570129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.570946 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.570950 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.571070 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.571141 (+ 71us) client_negotiation.cc:769] Sending connection context
0805 23:17:59.571212 (+ 71us) client_negotiation.cc:240] Negotiation successful
0805 23:17:59.571299 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":128,"threads_started":1}
I20260805 23:17:59.572060 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.565715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:49175 (local address 127.23.88.67:37365)
0805 23:17:59.565825 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.565829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.566253 (+ 424us) server_negotiation.cc:407] Connection header received
0805 23:17:59.566531 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.566534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.566583 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.566628 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:59.567231 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.568004 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.570253 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.570808 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.570932 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.570986 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.571841 (+ 855us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.571929 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:17:59.572697 5318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b41f332784481ea2053f7f86fcf4c3" candidate_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0688058f651b4466ac62dee86aca61ab" is_pre_election: true
I20260805 23:17:59.572883 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.565566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46031 (local address 127.23.88.65:40729)
0805 23:17:59.565705 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.565710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.567656 (+ 1946us) server_negotiation.cc:407] Connection header received
0805 23:17:59.567843 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.567849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.567902 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.567970 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:17:59.568527 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.569236 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.571601 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.572186 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.572287 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.572334 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.572689 (+ 355us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.572773 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:17:59.573057 5318 raft_consensus.cc:2454] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aef29f0c54a4b1e8b1b58937a726d26 in term 0.
I20260805 23:17:59.573279 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.565295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:40729 (local address 127.23.88.66:46031)
0805 23:17:59.567565 (+ 2270us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.567579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:17:59.567729 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:17:59.568074 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:17:59.568077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:17:59.568094 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:17:59.568370 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.568378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.569367 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.569370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.571223 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:17:59.571233 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.572436 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.572439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:17:59.572546 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.572566 (+ 20us) client_negotiation.cc:769] Sending connection context
0805 23:17:59.573056 (+ 490us) client_negotiation.cc:240] Negotiation successful
0805 23:17:59.573142 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2203,"mutex_wait_us":544,"thread_start_us":93,"threads_started":1}
I20260805 23:17:59.574020 5033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b41f332784481ea2053f7f86fcf4c3" candidate_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" is_pre_election: true
I20260805 23:17:59.574139 5105 leader_election.cc:302] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0688058f651b4466ac62dee86aca61ab, 2aef29f0c54a4b1e8b1b58937a726d26; no voters:
I20260805 23:17:59.574301 5033 raft_consensus.cc:2454] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aef29f0c54a4b1e8b1b58937a726d26 in term 0.
I20260805 23:17:59.574527 5412 raft_consensus.cc:2781] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:17:59.574630 5412 raft_consensus.cc:482] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:17:59.574697 5412 raft_consensus.cc:3037] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:59.576282 5412 raft_consensus.cc:504] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.576714 5412 leader_election.cc:288] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [CANDIDATE]: Term 1 election: Requested vote from peers 0688058f651b4466ac62dee86aca61ab (127.23.88.67:37365), 0f38cbea1ace4f029d63bc3eac5c781d (127.23.88.65:40729)
I20260805 23:17:59.577353 5318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b41f332784481ea2053f7f86fcf4c3" candidate_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0688058f651b4466ac62dee86aca61ab"
I20260805 23:17:59.577499 5318 raft_consensus.cc:3037] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:59.577538 5033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35b41f332784481ea2053f7f86fcf4c3" candidate_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f38cbea1ace4f029d63bc3eac5c781d"
I20260805 23:17:59.577647 5033 raft_consensus.cc:3037] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:17:59.579421 5033 raft_consensus.cc:2454] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aef29f0c54a4b1e8b1b58937a726d26 in term 1.
I20260805 23:17:59.579421 5318 raft_consensus.cc:2454] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aef29f0c54a4b1e8b1b58937a726d26 in term 1.
I20260805 23:17:59.579828 5105 leader_election.cc:302] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0688058f651b4466ac62dee86aca61ab, 2aef29f0c54a4b1e8b1b58937a726d26; no voters:
I20260805 23:17:59.580011 5412 raft_consensus.cc:2781] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:17:59.580260 5412 raft_consensus.cc:686] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 1 LEADER]: Becoming Leader. State: Replica: 2aef29f0c54a4b1e8b1b58937a726d26, State: Running, Role: LEADER
I20260805 23:17:59.580658 5412 consensus_queue.cc:237] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } }
I20260805 23:17:59.584677 4869 catalog_manager.cc:5495] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2aef29f0c54a4b1e8b1b58937a726d26 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aef29f0c54a4b1e8b1b58937a726d26" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 40337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 } health_report { overall_health: UNKNOWN } } }
Aug 05 23:17:59 dist-test-slave-vkjp krb5kdc[3818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for HTTP/127.23.88.84@KRBTEST.COM
I20260805 23:17:59.607164 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.601024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56526 (local address 127.23.88.66:40337)
0805 23:17:59.601141 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0805 23:17:59.601145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:17:59.601207 (+ 62us) server_negotiation.cc:407] Connection header received
0805 23:17:59.601411 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:17:59.601414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:17:59.601483 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:17:59.601594 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:17:59.601843 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.602531 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.603740 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:17:59.604095 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:17:59.604251 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:17:59.605506 (+ 1255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:17:59.606121 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:17:59.606245 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0805 23:17:59.606779 (+ 534us) server_negotiation.cc:299] Negotiation successful
0805 23:17:59.606852 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:17:59.616056 5033 raft_consensus.cc:1261] T 35b41f332784481ea2053f7f86fcf4c3 P 0f38cbea1ace4f029d63bc3eac5c781d [term 1 FOLLOWER]: Refusing update from remote peer 2aef29f0c54a4b1e8b1b58937a726d26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:17:59.616304 5318 raft_consensus.cc:1261] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 1 FOLLOWER]: Refusing update from remote peer 2aef29f0c54a4b1e8b1b58937a726d26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:17:59.617115 5412 consensus_queue.cc:1035] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0688058f651b4466ac62dee86aca61ab" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 37365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:17:59.617774 5418 consensus_queue.cc:1035] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f38cbea1ace4f029d63bc3eac5c781d" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 40729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:17:59.634778 5421 mvcc.cc:205] Tried to move back new op lower bound from 7315340818897887232 to 7315340818766204928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340818897887232}]
I20260805 23:17:59.637570 5422 mvcc.cc:205] Tried to move back new op lower bound from 7315340818897887232 to 7315340818766204928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340818897887232}]
I20260805 23:17:59.643114 5423 mvcc.cc:205] Tried to move back new op lower bound from 7315340818897887232 to 7315340818766204928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340818897887232}]
I20260805 23:17:59.656487 4869 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:17:59.656705 4869 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:17:59.660131 4869 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbaaa8b486b943a0bd67e49ddec8a101: Sending DeleteTablet for 3 replicas of tablet 35b41f332784481ea2053f7f86fcf4c3
I20260805 23:17:59.662320 5141 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:59 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:56512
I20260805 23:17:59.662925 5283 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:59 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:50974
I20260805 23:17:59.663480 5011 tablet_service.cc:1514] Processing DeleteTablet for tablet 35b41f332784481ea2053f7f86fcf4c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:17:59 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:40058
I20260805 23:17:59.665505 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4909
I20260805 23:17:59.666654 5430 tablet_replica.cc:330] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: stopping tablet replica
I20260805 23:17:59.667013 5430 raft_consensus.cc:2229] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:17:59.667479 5430 raft_consensus.cc:2258] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:17:59.671412 5431 tablet_replica.cc:330] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: stopping tablet replica
I20260805 23:17:59.673171 5430 ts_tablet_manager.cc:1905] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:17:59.673626 5431 raft_consensus.cc:2229] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:17:59.674083 5431 raft_consensus.cc:2258] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:17:59.681267 5431 ts_tablet_manager.cc:1905] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:17:59.685235 5431 ts_tablet_manager.cc:1918] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:17:59.685470 5431 log.cc:1196] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/35b41f332784481ea2053f7f86fcf4c3
I20260805 23:17:59.685969 5431 ts_tablet_manager.cc:1939] T 35b41f332784481ea2053f7f86fcf4c3 P 0688058f651b4466ac62dee86aca61ab: Deleting consensus metadata
I20260805 23:17:59.687276 5430 ts_tablet_manager.cc:1918] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:17:59.687513 5430 log.cc:1196] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/35b41f332784481ea2053f7f86fcf4c3
I20260805 23:17:59.687685 4852 catalog_manager.cc:4841] TS 0688058f651b4466ac62dee86aca61ab (127.23.88.67:37365): tablet 35b41f332784481ea2053f7f86fcf4c3 (table test-table [id=37431c9b9e2946fa802ff9044a07c5a2]) successfully deleted
I20260805 23:17:59.687963 5430 ts_tablet_manager.cc:1939] T 35b41f332784481ea2053f7f86fcf4c3 P 2aef29f0c54a4b1e8b1b58937a726d26: Deleting consensus metadata
I20260805 23:17:59.693682 4852 catalog_manager.cc:4841] TS 2aef29f0c54a4b1e8b1b58937a726d26 (127.23.88.66:40337): tablet 35b41f332784481ea2053f7f86fcf4c3 (table test-table [id=37431c9b9e2946fa802ff9044a07c5a2]) successfully deleted
W20260805 23:17:59.694154 4856 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:40729 (error 108)
I20260805 23:17:59.695731 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5083
I20260805 23:17:59.700659 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:17:59.700275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:40729 (local address 127.0.0.1:40072)
0805 23:17:59.700485 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 23:17:59.700562 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:40729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260805 23:17:59.701023 4856 catalog_manager.cc:4584] TS 0f38cbea1ace4f029d63bc3eac5c781d (127.23.88.65:40729): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35b41f332784481ea2053f7f86fcf4c3: Network error: Client connection negotiation failed: client connection to 127.23.88.65:40729: connect: Connection refused (error 111)
I20260805 23:17:59.706928 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5228
I20260805 23:17:59.716616 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 4834
I20260805 23:17:59.729483 23905 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260805 23:18:00.075253 23905 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260805 23:18:00.075378 23905 mini_ranger.cc:66] Stopping Ranger...
I20260805 23:18:00.421043 23905 mini_ranger.cc:68] Stopped Ranger
2026-08-05 23:18:00.421 UTC [3849] LOG: received smart shutdown request
2026-08-05 23:18:00.426 UTC [3849] LOG: background worker "logical replication launcher" (PID 3858) exited with exit code 1
2026-08-05 23:18:00.426 UTC [3853] LOG: shutting down
2026-08-05 23:18:00.633 UTC [3849] LOG: database system is shut down
2026-08-05T23:18:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50666 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:18:00 dist-test-slave-vkjp krb5kdc[5443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:18:00 dist-test-slave-vkjp krb5kdc[5443](info): set up 2 sockets
Aug 05 23:18:00 dist-test-slave-vkjp krb5kdc[5443](info): commencing operation
krb5kdc: starting...
W20260805 23:18:01.850143 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:18:01 dist-test-slave-vkjp krb5kdc[5443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:18:01.983541 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37337
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:02.100014 5457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:02.100353 5457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:02.100483 5457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:02.104816 5457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:02.104897 5457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:02.104919 5457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:02.104936 5457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:02.104952 5457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:02.110651 5457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:02.111941 5457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:02.112972 5457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:02.122118 5466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.125509 5463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.125769 5464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:02.127056 5457 server_base.cc:1034] running on GCE node
I20260805 23:18:02.127877 5457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:02.129171 5457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:02.130376 5457 hybrid_clock.cc:648] HybridClock initialized: now 1785971882130361 us; error 36 us; skew 500 ppm
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:02.138329 5457 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:02.139534 5457 webserver.cc:460] Webserver started at http://127.23.88.126:33857/ using document root <none> and password file <none>
I20260805 23:18:02.140154 5457 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:02.140251 5457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:02.140522 5457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:02.142745 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "482a3f9806074645baba53e8618b2bc3"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "92637f21a8e04603dda58d0419295f19"
server_key_iv: "3cf386d08d0522c9a3e94f327757493a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.143374 5457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "482a3f9806074645baba53e8618b2bc3"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "92637f21a8e04603dda58d0419295f19"
server_key_iv: "3cf386d08d0522c9a3e94f327757493a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.147631 5457 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:18:02.150465 5474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:02.151901 5457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:02.152067 5457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "482a3f9806074645baba53e8618b2bc3"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "92637f21a8e04603dda58d0419295f19"
server_key_iv: "3cf386d08d0522c9a3e94f327757493a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.152192 5457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:02.168166 5457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:02.171850 5457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:02.172086 5457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:02.180037 5457 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:37337
I20260805 23:18:02.180050 5528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:37337 every 8 connection(s)
I20260805 23:18:02.181078 5457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:02.184149 5529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:02.186501 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5457
I20260805 23:18:02.186653 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:18:02.186957 23905 external_mini_cluster.cc:1428] Setting key b849550b82ca6c29f78fa72e33037533
I20260805 23:18:02.189149 5529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.88.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.125@KRBTEST.COM" created.
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:02.205791 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.191543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37337 (local address 127.0.0.1:36326)
0805 23:18:02.192066 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.192121 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.192982 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.193787 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.193791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.194144 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.194410 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.194423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.196233 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.196235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.197104 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.197115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.197786 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.197790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.197902 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.198530 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.198546 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.200312 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.202448 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.202461 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.202475 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.202814 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.203187 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.203193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.203195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.203364 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.204040 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.204049 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.204251 (+ 202us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.204483 (+ 232us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.204896 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":37,"thread_start_us":155,"threads_started":1}
I20260805 23:18:02.205796 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.191544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36326 (local address 127.23.88.126:37337)
0805 23:18:02.192201 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.192214 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.192264 (+ 50us) server_negotiation.cc:407] Connection header received
0805 23:18:02.193308 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.193318 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.193576 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.193798 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.194569 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.196074 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.197228 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.197600 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.197732 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.200494 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.200506 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.200524 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.200559 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.202322 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.202933 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.202937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.202939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.203044 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.203477 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.203479 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.203481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.203907 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.204013 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.204572 (+ 559us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.204800 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":156,"threads_started":1}
I20260805 23:18:02.208609 5529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.209438 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.209148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34304)
0805 23:18:02.209273 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.209348 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260805 23:18:02.209638 5475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
I20260805 23:18:02.210597 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.210387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34312)
0805 23:18:02.210497 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.210544 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260805 23:18:02.210902 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34517: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
I20260805 23:18:02.218760 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:34517
--webserver_interface=127.23.88.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:02.260166 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } attempt: 1
I20260805 23:18:02.261134 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.260796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34324)
0805 23:18:02.260970 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.261052 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260805 23:18:02.261932 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.261689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34330)
0805 23:18:02.261798 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.261852 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260805 23:18:02.262064 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34517: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
I20260805 23:18:02.330307 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } attempt: 2
I20260805 23:18:02.331346 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.330913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34340)
0805 23:18:02.331142 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.331219 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260805 23:18:02.332155 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.331907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34342)
0805 23:18:02.332046 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.332094 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260805 23:18:02.332299 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34517: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34517: connect: Connection refused (error 111)
W20260805 23:18:02.335701 5536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:02.336006 5536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:02.336081 5536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:02.340375 5536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:02.340442 5536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:02.340463 5536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:02.340499 5536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:02.340530 5536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:02.346199 5536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:34517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.88.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:02.347461 5536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:02.348521 5536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:02.357414 5543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.357401 5545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.357396 5542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:02.357808 5536 server_base.cc:1034] running on GCE node
I20260805 23:18:02.358397 5536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:02.359710 5536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:02.360919 5536 hybrid_clock.cc:648] HybridClock initialized: now 1785971882360900 us; error 35 us; skew 500 ppm
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:02.363931 5536 init.cc:377] Logged in from keytab as kudu/127.23.88.125@KRBTEST.COM (short username kudu)
I20260805 23:18:02.365001 5536 webserver.cc:460] Webserver started at http://127.23.88.125:45015/ using document root <none> and password file <none>
I20260805 23:18:02.365576 5536 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:02.365666 5536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:02.365912 5536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:02.368109 5536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data/instance:
uuid: "093e55dc279849fe9106a60d9b7abace"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "c8a77de2e3cd1e21595b706e2fb7878c"
server_key_iv: "ca63384ba73e64756481337d4e1e054a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.368739 5536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal/instance:
uuid: "093e55dc279849fe9106a60d9b7abace"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "c8a77de2e3cd1e21595b706e2fb7878c"
server_key_iv: "ca63384ba73e64756481337d4e1e054a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.372946 5536 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:18:02.375808 5553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:02.377573 5536 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:02.377765 5536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "093e55dc279849fe9106a60d9b7abace"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "c8a77de2e3cd1e21595b706e2fb7878c"
server_key_iv: "ca63384ba73e64756481337d4e1e054a"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.377915 5536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:02.390769 5536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:02.394271 5536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:02.394527 5536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:02.402063 5536 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.125:34517
I20260805 23:18:02.402091 5606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.125:34517 every 8 connection(s)
I20260805 23:18:02.403162 5536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
I20260805 23:18:02.405802 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5536
I20260805 23:18:02.405963 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-1/wal/instance
I20260805 23:18:02.406316 23905 external_mini_cluster.cc:1428] Setting key e28d57c8c9e7340b73715a44059dada6
I20260805 23:18:02.406750 5607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:02.411830 5607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.88.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.124@KRBTEST.COM" created.
I20260805 23:18:02.423527 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } attempt: 3
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:02.429839 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.414868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.23.88.126:37337)
0805 23:18:02.415068 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.415074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.416178 (+ 1104us) server_negotiation.cc:407] Connection header received
0805 23:18:02.417355 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.417359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.417419 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.417511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.418946 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.419751 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.421035 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.421440 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.421584 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.425224 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.425231 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.425237 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.425290 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.427267 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.427920 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.427923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.427925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.428031 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.428558 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.428562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.428563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.428800 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.428905 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.429515 (+ 610us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.429647 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260805 23:18:02.430917 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.415376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37337 (local address 127.0.0.1:36342)
0805 23:18:02.415970 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.416032 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.417108 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.417693 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.417700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.418064 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.418746 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.418763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.419937 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.419940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.420880 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.420891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.421665 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.421670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.421915 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.422686 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.422705 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.425014 (+ 2309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.427415 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.427427 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.427443 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.427800 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.428140 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.428145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.428150 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.428415 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.428923 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.428930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.429248 (+ 318us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.429459 (+ 211us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.429794 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":190,"threads_started":1}
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:18:02.432734 5607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.436523 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.424114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34346)
0805 23:18:02.424377 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.424391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.424469 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.425381 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.425384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.425623 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.425790 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.425799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.428258 (+ 2459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.428262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.428949 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.428960 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.429712 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.429715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.429844 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.430380 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.430401 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.432256 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.434721 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.434725 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.434727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.435009 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.435351 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.435355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.435357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.435450 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.436014 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.436017 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.436107 (+ 90us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.436192 (+ 85us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.436361 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260805 23:18:02.436770 5615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.424435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34346 (local address 127.23.88.125:34517)
0805 23:18:02.424896 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.424916 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.424946 (+ 30us) server_negotiation.cc:407] Connection header received
0805 23:18:02.424998 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.425004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.425233 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.425381 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.425953 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.428112 (+ 2159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.429070 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.429568 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.429695 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.432425 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.432433 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.432444 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.432468 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.434587 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.435164 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.435176 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.435177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.435255 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.435537 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.435540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.435541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.435900 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.436041 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.436314 (+ 273us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.436518 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":197,"threads_started":1}
I20260805 23:18:02.439117 5529 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:18:02.440060 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.439722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51790)
0805 23:18:02.439888 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.439964 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260805 23:18:02.440438 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:43595
--webserver_interface=127.23.88.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:02.441308 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.440659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51806)
0805 23:18:02.440862 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.440939 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260805 23:18:02.441455 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.443583 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.433347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34358)
0805 23:18:02.433498 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.433514 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.433617 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.434071 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.434074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.434452 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.434585 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.434591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.435656 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.435660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.436426 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.436436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.437019 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.437022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.437112 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.437625 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.437640 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.439923 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.441990 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.441994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.441997 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.442242 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.442516 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.442520 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.442522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.442606 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.443007 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.443010 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.443124 (+ 114us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.443241 (+ 117us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.443408 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260805 23:18:02.443629 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.433496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.23.88.125:34517)
0805 23:18:02.433862 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.433869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.433881 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:18:02.433909 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.433910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.433943 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.434025 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.434721 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.435555 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.436551 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.436903 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.437018 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.440086 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.440090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.440093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.440127 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.441872 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.442347 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.442350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.442351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.442424 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.442704 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.442707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.442709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.442914 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.443019 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.443302 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.443470 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":308,"thread_start_us":170,"threads_started":1}
I20260805 23:18:02.444708 5607 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.445602 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.445262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51814)
0805 23:18:02.445440 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.445537 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260805 23:18:02.445858 5555 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.446914 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.446669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51816)
0805 23:18:02.446793 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.446855 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260805 23:18:02.447073 5607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.472633 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 1
I20260805 23:18:02.473650 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.473274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51828)
0805 23:18:02.473465 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.473546 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260805 23:18:02.474491 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.474229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51838)
0805 23:18:02.474354 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.474417 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260805 23:18:02.474676 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.496435 5607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 1
I20260805 23:18:02.497390 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.497061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51850)
0805 23:18:02.497224 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.497300 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:18:02.498230 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.498001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51862)
0805 23:18:02.498121 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.498177 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260805 23:18:02.498418 5607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.549875 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 2
I20260805 23:18:02.550869 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.550490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51868)
0805 23:18:02.550693 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.550768 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260805 23:18:02.551818 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.551555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51884)
0805 23:18:02.551681 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.551744 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:18:02.551971 5529 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
W20260805 23:18:02.561628 5617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:02.561939 5617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:02.561982 5617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:02.566298 5617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:02.566375 5617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:02.566401 5617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:02.566419 5617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:02.566471 5617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:02.566586 5607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 2
I20260805 23:18:02.567562 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.567207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51886)
0805 23:18:02.567419 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.567481 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260805 23:18:02.568455 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.568209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51898)
0805 23:18:02.568331 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.568388 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260805 23:18:02.568665 5607 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:43595: Network error: Client connection negotiation failed: client connection to 127.23.88.124:43595: connect: Connection refused (error 111)
I20260805 23:18:02.573773 5617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37337,127.23.88.125:34517,127.23.88.124:43595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:43595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.88.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:02.575021 5617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:02.576149 5617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:02.585476 5628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.585491 5625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:02.585505 5626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:02.585536 5617 server_base.cc:1034] running on GCE node
I20260805 23:18:02.586568 5617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:02.587837 5617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:02.589020 5617 hybrid_clock.cc:648] HybridClock initialized: now 1785971882589007 us; error 27 us; skew 500 ppm
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:02.592089 5617 init.cc:377] Logged in from keytab as kudu/127.23.88.124@KRBTEST.COM (short username kudu)
I20260805 23:18:02.593015 5617 webserver.cc:460] Webserver started at http://127.23.88.124:32811/ using document root <none> and password file <none>
I20260805 23:18:02.593550 5617 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:02.593611 5617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:02.593784 5617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:02.595978 5617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data/instance:
uuid: "f08cf47c871a40f38b93184fac645670"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "3636948ace091d948d4942e11bc4e239"
server_key_iv: "46e3c91d977d2c6d7512e41169bbaa30"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.596529 5617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal/instance:
uuid: "f08cf47c871a40f38b93184fac645670"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "3636948ace091d948d4942e11bc4e239"
server_key_iv: "46e3c91d977d2c6d7512e41169bbaa30"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.600560 5617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:18:02.603433 5635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:02.604908 5617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:02.605067 5617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "f08cf47c871a40f38b93184fac645670"
format_stamp: "Formatted at 2026-08-05 23:18:02 on dist-test-slave-vkjp"
server_key: "3636948ace091d948d4942e11bc4e239"
server_key_iv: "46e3c91d977d2c6d7512e41169bbaa30"
server_key_version: "encryptionkey@0"
I20260805 23:18:02.605208 5617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:02.637177 5617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:02.641224 5617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:02.641474 5617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:02.649804 5617 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.124:43595
I20260805 23:18:02.649816 5689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.124:43595 every 8 connection(s)
I20260805 23:18:02.650977 5617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
I20260805 23:18:02.651153 5529 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 3
I20260805 23:18:02.654170 5690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:02.659000 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5617
I20260805 23:18:02.659229 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785971715702001-23905-0/minicluster-data/master-2/wal/instance
I20260805 23:18:02.659011 5690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.659580 23905 external_mini_cluster.cc:1428] Setting key 1c1cbea0e42337bea76368cb31eec813
I20260805 23:18:02.659850 5607 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } attempt: 3
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:18:02.666271 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.651794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51902)
0805 23:18:02.651954 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.651972 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.652079 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.654538 (+ 2459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.654542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.654761 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.654894 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.654903 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.656513 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.656517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.657355 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.657371 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.658195 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.658199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.658288 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.658973 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.658990 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.661618 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.664124 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.664131 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.664136 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.664427 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.664868 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.664873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.664875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.664991 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.665739 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.665742 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.665839 (+ 97us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.665942 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.666091 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:18:02.668226 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.652542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51902 (local address 127.23.88.124:43595)
0805 23:18:02.652990 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.653010 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.653061 (+ 51us) server_negotiation.cc:407] Connection header received
0805 23:18:02.653785 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.653804 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.654176 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.654550 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.655027 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.656369 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.657545 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.658050 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.658187 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.661843 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.661850 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.661866 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.661897 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.663993 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.664572 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.664579 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.664585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.664747 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.665118 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.665121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.665122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.665616 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.666637 (+ 1021us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.666799 (+ 162us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.667154 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":165,"threads_started":1}
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:18:02.673367 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.661329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.23.88.126:37337)
0805 23:18:02.661458 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.661463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.661859 (+ 396us) server_negotiation.cc:407] Connection header received
0805 23:18:02.662087 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.662090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.662138 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.662239 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.663145 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.663922 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.665187 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.665540 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.665618 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.669150 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.669156 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.669159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.669193 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.670959 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.671689 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.671694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.671698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.671791 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.672158 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.672161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.672164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.672486 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.672581 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.673041 (+ 460us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.673175 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:18:02.673537 5529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: Bootstrap starting.
I20260805 23:18:02.674116 5613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.660350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51914)
0805 23:18:02.660471 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.660488 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.660576 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.661630 (+ 1054us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.661634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.661832 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.661983 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.661992 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.663255 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.663259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.664119 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.664129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.664737 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.664741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.664833 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.665326 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.665342 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.667291 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.672335 (+ 5044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.672338 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.672343 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.672610 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.672948 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.672951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.672953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.673048 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.673626 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.673629 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.673734 (+ 105us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.673818 (+ 84us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.673965 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260805 23:18:02.674270 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.661281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37337 (local address 127.0.0.1:36350)
0805 23:18:02.661719 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.661787 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.661944 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.662385 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.662393 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.662805 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.662978 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.662987 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.664068 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.664071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.665040 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.665052 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.665774 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.665779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.665919 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.666592 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.666612 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.668984 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.671116 (+ 2132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.671127 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.671133 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.671561 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.671933 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.671937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.671940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.672036 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.672651 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.672657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.672795 (+ 138us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.673873 (+ 1078us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.674068 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
I20260805 23:18:02.674608 5694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.660761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.23.88.124:43595)
0805 23:18:02.661159 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.661169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.662481 (+ 1312us) server_negotiation.cc:407] Connection header received
0805 23:18:02.663119 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.663122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.663170 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.663230 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.665119 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.665833 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.667100 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.667413 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.667544 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.670776 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.670780 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.670782 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.670810 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.672641 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.673138 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.673144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.673146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.673245 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.673570 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.673575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.673577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.673770 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.673928 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.674292 (+ 364us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.674419 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":321,"thread_start_us":182,"threads_started":1}
I20260805 23:18:02.675467 5690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.676640 5529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:02.677110 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.661094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51914 (local address 127.23.88.124:43595)
0805 23:18:02.661340 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.661351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.661364 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:18:02.661404 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.661407 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.661454 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.662113 (+ 659us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.662237 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.663074 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.664245 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.664599 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.667461 (+ 2862us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.667662 (+ 201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.667667 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.667670 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.667693 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.672174 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.672747 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.672750 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.672751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.672836 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.673165 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.673168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.673170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.673413 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.676643 (+ 3230us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.676730 (+ 87us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.676906 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":188,"spinlock_wait_cycles":4352,"thread_start_us":106,"threads_started":1}
I20260805 23:18:02.677932 5529 log.cc:824] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:02.680485 5529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: No bootstrap required, opened a new log
I20260805 23:18:02.681387 5607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: Bootstrap starting.
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:02.685176 5529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.685459 5529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:02.685514 5529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482a3f9806074645baba53e8618b2bc3, State: Initialized, Role: FOLLOWER
I20260805 23:18:02.685529 5607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:02.686348 5529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.686465 5607 log.cc:824] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:02.687057 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.676354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.23.88.126:37337)
0805 23:18:02.676477 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.676483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.676497 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:18:02.676578 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.676584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.676635 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.676705 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.677554 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.678226 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.679324 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.679649 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.679727 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.682906 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.682910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.682912 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.682937 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.684886 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685522 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.685526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.685528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.685637 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685973 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.685976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.685978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.686210 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.686372 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.686754 (+ 382us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.686889 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:18:02.687057 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.676070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.23.88.125:34517)
0805 23:18:02.676178 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.676184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.676235 (+ 51us) server_negotiation.cc:407] Connection header received
0805 23:18:02.676399 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.676402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.676453 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.676507 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.677171 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.677975 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.679128 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.679489 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.679565 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.682896 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.682901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.682904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.682936 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.685005 (+ 2069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685522 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.685526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.685528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.685637 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685976 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.685979 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.685981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.686262 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.686327 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.686723 (+ 396us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.686912 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:18:02.687599 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.676015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34517 (local address 127.0.0.1:34364)
0805 23:18:02.676177 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.676191 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.676276 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.676650 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.676655 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.676885 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.677019 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.677027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.678112 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.678115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.678956 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.678967 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.679743 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.679746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.679861 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.680443 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.680456 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.682681 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.685135 (+ 2454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.685145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.685404 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.685763 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.685766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.685768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.685852 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.686407 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.686413 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.686524 (+ 111us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.687228 (+ 704us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.687415 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260805 23:18:02.688211 5529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:02.688690 5690 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:18:02.693336 5699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.693650 5699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: This master's current role is: FOLLOWER
W20260805 23:18:02.694082 5710 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:18:02.694248 5710 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971882, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
Aug 05 23:18:02 dist-test-slave-vkjp krb5kdc[5443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:18:02.694073 5457 master_runner.cc:186] Error getting master registration for 127.23.88.125:34517: OK, instance_id { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" instance_seqno: 1785971882399828 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:18:02.699237 5457 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:18:02.699676 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.689371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:43595 (local address 127.0.0.1:51930)
0805 23:18:02.689484 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:02.689498 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:02.689596 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:02.690113 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:02.690117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:02.690318 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:02.690461 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.690471 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.691468 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.691470 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.692202 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:02.692217 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.692843 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.692846 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:02.692925 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.693933 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:02.693949 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:02.695384 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:02.697184 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.697187 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.697189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.697418 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.697727 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:02.697731 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:02.697733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:02.698206 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:02.699181 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:02.699184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:02.699307 (+ 123us) client_negotiation.cc:769] Sending connection context
0805 23:18:02.699427 (+ 120us) client_negotiation.cc:240] Negotiation successful
0805 23:18:02.699548 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260805 23:18:02.701792 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.689115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.23.88.125:34517)
0805 23:18:02.689230 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.689233 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.689667 (+ 434us) server_negotiation.cc:407] Connection header received
0805 23:18:02.689860 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.689863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.689908 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.689988 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.691170 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.691856 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.692895 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.693559 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.693637 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.697332 (+ 3695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.697336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.697339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.697371 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.699014 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.700303 (+ 1289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.700306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.700308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.700406 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.700702 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.700706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.700708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.700909 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.701122 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.701332 (+ 210us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.701466 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:18:02.701789 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.689415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.23.88.124:43595)
0805 23:18:02.689507 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.689513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.689680 (+ 167us) server_negotiation.cc:407] Connection header received
0805 23:18:02.689707 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.689710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.689754 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.689835 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.690561 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.691350 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.692330 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.692640 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.692730 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.695519 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.695523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.695526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.695549 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.697057 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.697517 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.697522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.697524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.697629 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.698355 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.698361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.698363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.698578 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.698647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.699521 (+ 874us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.699665 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:18:02.703647 5607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: No bootstrap required, opened a new log
I20260805 23:18:02.705394 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: Bootstrap starting.
I20260805 23:18:02.707178 5607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.707507 5607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:02.707607 5607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 093e55dc279849fe9106a60d9b7abace, State: Initialized, Role: FOLLOWER
I20260805 23:18:02.708164 5607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.708536 5690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:02.709679 5690 log.cc:824] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:02.709824 5607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:02.710177 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:18:02.712251 5690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: No bootstrap required, opened a new log
I20260805 23:18:02.716746 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.717078 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.717923 5690 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.718274 5690 raft_consensus.cc:374] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:02.718392 5690 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08cf47c871a40f38b93184fac645670, State: Initialized, Role: FOLLOWER
I20260805 23:18:02.719261 5690 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.720768 5690 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:18:02.721562 5729 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:18:02.721714 5729 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:18:02.719362 5536 master_runner.cc:186] Error getting master registration for 127.23.88.124:43595: OK, instance_id { permanent_uuid: "f08cf47c871a40f38b93184fac645670" instance_seqno: 1785971882647411 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:18:02.723007 5536 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:18:02.729787 5617 master_runner.cc:263] No leader master found from master f08cf47c871a40f38b93184fac645670
I20260805 23:18:02.729877 5617 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260805 23:18:02.730840 5747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:18:02.730945 5747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:18:02.731781 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.732017 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.739257 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.714065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.23.88.126:37337)
0805 23:18:02.714199 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.714204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.723778 (+ 9574us) server_negotiation.cc:407] Connection header received
0805 23:18:02.724001 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.724005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.724059 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.724111 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.725148 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.725941 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.727407 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.727711 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.727802 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.734923 (+ 7121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.734928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.734931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.734967 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.737028 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.737599 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.737604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.737606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.737708 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.738080 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.738085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.738087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.738334 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.738476 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.738888 (+ 412us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.739027 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:18:02.745579 5616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.715352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34394 (local address 127.23.88.125:34517)
0805 23:18:02.715468 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.715472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.729201 (+ 13729us) server_negotiation.cc:407] Connection header received
0805 23:18:02.729410 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.729414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.729464 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.729513 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.730501 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.731241 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.733576 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.733910 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.734008 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.738530 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.738533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.738536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.738568 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.740286 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.741186 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.741190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.741192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.741291 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.741901 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.741904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.741906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.742134 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.742237 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.745233 (+ 2996us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.745400 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:18:02.747751 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:02.714621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51936 (local address 127.23.88.124:43595)
0805 23:18:02.714740 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:02.714745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:02.731485 (+ 16740us) server_negotiation.cc:407] Connection header received
0805 23:18:02.731594 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:02.731597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:02.731644 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:02.731717 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:02.732704 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.733464 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.735549 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:02.735900 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:02.735973 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:02.742619 (+ 6646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:02.742623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:02.742626 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:02.742661 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:02.744590 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.746074 (+ 1484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.746080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.746082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.746220 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:02.746595 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:02.746600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:02.746602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:02.746879 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:02.747033 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:02.747468 (+ 435us) server_negotiation.cc:299] Negotiation successful
0805 23:18:02.747592 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260805 23:18:02.893167 5641 tablet.cc:2369] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:18:02.896318 5559 tablet.cc:2369] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:18:02.924115 5482 tablet.cc:2369] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:18:02.947728 5699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:02.948022 5699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.949100 5699 leader_election.cc:288] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 093e55dc279849fe9106a60d9b7abace (127.23.88.125:34517), f08cf47c871a40f38b93184fac645670 (127.23.88.124:43595)
I20260805 23:18:02.949708 5580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "482a3f9806074645baba53e8618b2bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093e55dc279849fe9106a60d9b7abace" is_pre_election: true
I20260805 23:18:02.949772 5663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "482a3f9806074645baba53e8618b2bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08cf47c871a40f38b93184fac645670" is_pre_election: true
I20260805 23:18:02.950039 5580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482a3f9806074645baba53e8618b2bc3 in term 0.
I20260805 23:18:02.950088 5663 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482a3f9806074645baba53e8618b2bc3 in term 0.
I20260805 23:18:02.950512 5475 leader_election.cc:302] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093e55dc279849fe9106a60d9b7abace, 482a3f9806074645baba53e8618b2bc3; no voters:
I20260805 23:18:02.950791 5699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:18:02.950882 5699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:02.950937 5699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:02.952387 5699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.952816 5699 leader_election.cc:288] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [CANDIDATE]: Term 1 election: Requested vote from peers 093e55dc279849fe9106a60d9b7abace (127.23.88.125:34517), f08cf47c871a40f38b93184fac645670 (127.23.88.124:43595)
I20260805 23:18:02.953114 5580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "482a3f9806074645baba53e8618b2bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "093e55dc279849fe9106a60d9b7abace"
I20260805 23:18:02.953244 5663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "482a3f9806074645baba53e8618b2bc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f08cf47c871a40f38b93184fac645670"
I20260805 23:18:02.953260 5580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:02.953379 5663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:02.954376 5580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482a3f9806074645baba53e8618b2bc3 in term 1.
I20260805 23:18:02.954412 5663 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482a3f9806074645baba53e8618b2bc3 in term 1.
I20260805 23:18:02.954679 5475 leader_election.cc:302] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 093e55dc279849fe9106a60d9b7abace, 482a3f9806074645baba53e8618b2bc3; no voters:
I20260805 23:18:02.954852 5699 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:02.955132 5699 raft_consensus.cc:686] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [term 1 LEADER]: Becoming Leader. State: Replica: 482a3f9806074645baba53e8618b2bc3, State: Running, Role: LEADER
I20260805 23:18:02.955497 5699 consensus_queue.cc:237] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } }
I20260805 23:18:02.957558 5749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482a3f9806074645baba53e8618b2bc3. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.957789 5749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:02.958199 5751 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:02.959059 5751 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:02.963905 5580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [term 1 FOLLOWER]: Refusing update from remote peer 482a3f9806074645baba53e8618b2bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:02.963915 5663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [term 1 FOLLOWER]: Refusing update from remote peer 482a3f9806074645baba53e8618b2bc3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:02.964628 5699 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:02.964855 5749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:02.968855 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482a3f9806074645baba53e8618b2bc3. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.968844 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: SysCatalogTable state changed. Reason: New leader 482a3f9806074645baba53e8618b2bc3. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.969002 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.969002 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.972107 5699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.972329 5699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:02.972541 5699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.972644 5699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:02.972709 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.972838 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.972913 5755 mvcc.cc:205] Tried to move back new op lower bound from 7315340832612622336 to 7315340832588783616. Current Snapshot: MvccSnapshot[applied={T|T < 7315340832612622336}]
I20260805 23:18:02.973739 5712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "482a3f9806074645baba53e8618b2bc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482a3f9806074645baba53e8618b2bc3" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37337 } } peers { permanent_uuid: "093e55dc279849fe9106a60d9b7abace" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34517 } } peers { permanent_uuid: "f08cf47c871a40f38b93184fac645670" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 43595 } } }
I20260805 23:18:02.973863 5712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:18:02.974625 5751 catalog_manager.cc:1347] Generated new cluster ID: cf381e067ad44fe2833395f017fd3164
I20260805 23:18:02.974701 5751 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:02.993947 5751 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:18:02.995350 5751 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:03.005146 5751 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 482a3f9806074645baba53e8618b2bc3: Generated new TSK 0
I20260805 23:18:03.006249 5751 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:18:03.723300 5729 catalog_manager.cc:1259] Loaded cluster ID: cf381e067ad44fe2833395f017fd3164
I20260805 23:18:03.723408 5729 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: loading cluster ID for follower catalog manager: success
I20260805 23:18:03.725098 5729 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: acquiring CA information for follower catalog manager: success
I20260805 23:18:03.726310 5729 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 093e55dc279849fe9106a60d9b7abace: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:18:03.732304 5747 catalog_manager.cc:1259] Loaded cluster ID: cf381e067ad44fe2833395f017fd3164
I20260805 23:18:03.732399 5747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: loading cluster ID for follower catalog manager: success
I20260805 23:18:03.734213 5747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: acquiring CA information for follower catalog manager: success
I20260805 23:18:03.735584 5747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f08cf47c871a40f38b93184fac645670: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:18:04.067601 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5457
I20260805 23:18:04.075309 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5536
I20260805 23:18:04.082459 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5617
2026-08-05T23:18:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3329 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:18:04 dist-test-slave-vkjp krb5kdc[5769](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:18:04 dist-test-slave-vkjp krb5kdc[5769](info): set up 2 sockets
Aug 05 23:18:04 dist-test-slave-vkjp krb5kdc[5769](info): commencing operation
krb5kdc: starting...
W20260805 23:18:05.154297 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:18:05.295789 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42005
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:05.414384 5784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:05.414752 5784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:05.414827 5784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:05.419114 5784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:05.419238 5784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:05.419275 5784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:05.419320 5784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:05.419356 5784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:05.425143 5784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:05.426371 5784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:05.427506 5784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:05.436604 5789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:05.436730 5790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:05.436604 5792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:05.436812 5784 server_base.cc:1034] running on GCE node
I20260805 23:18:05.437508 5784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:05.438623 5784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:05.439811 5784 hybrid_clock.cc:648] HybridClock initialized: now 1785971885439776 us; error 45 us; skew 500 ppm
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5769](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971885, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:05.443130 5784 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:05.444334 5784 webserver.cc:460] Webserver started at http://127.23.88.126:45123/ using document root <none> and password file <none>
I20260805 23:18:05.445036 5784 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:05.445108 5784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:05.445376 5784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:05.447939 5784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "982c8bc7f60741519afeef9c8276d962"
format_stamp: "Formatted at 2026-08-05 23:18:05 on dist-test-slave-vkjp"
server_key: "c0026c1af500b95974314d84b1337d44"
server_key_iv: "e650c818b1981be7a47c20d18b2f32fd"
server_key_version: "encryptionkey@0"
I20260805 23:18:05.448632 5784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "982c8bc7f60741519afeef9c8276d962"
format_stamp: "Formatted at 2026-08-05 23:18:05 on dist-test-slave-vkjp"
server_key: "c0026c1af500b95974314d84b1337d44"
server_key_iv: "e650c818b1981be7a47c20d18b2f32fd"
server_key_version: "encryptionkey@0"
I20260805 23:18:05.453112 5784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:18:05.456254 5800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:05.457929 5784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260805 23:18:05.458065 5784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "982c8bc7f60741519afeef9c8276d962"
format_stamp: "Formatted at 2026-08-05 23:18:05 on dist-test-slave-vkjp"
server_key: "c0026c1af500b95974314d84b1337d44"
server_key_iv: "e650c818b1981be7a47c20d18b2f32fd"
server_key_version: "encryptionkey@0"
I20260805 23:18:05.458247 5784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:05.485086 5784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:05.488946 5784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:05.489208 5784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:05.498701 5784 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42005
I20260805 23:18:05.498806 5854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42005 every 8 connection(s)
I20260805 23:18:05.500053 5784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:05.503327 5855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:05.503800 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5784
I20260805 23:18:05.504000 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:18:05.504328 23905 external_mini_cluster.cc:1428] Setting key ea284630df2a93735e1b67ae9b19576e
I20260805 23:18:05.510736 5855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962: Bootstrap starting.
I20260805 23:18:05.513830 5855 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:05.514775 5855 log.cc:824] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5769](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:05.517150 5855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962: No bootstrap required, opened a new log
I20260805 23:18:05.522771 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:05.505824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.23.88.126:42005)
0805 23:18:05.506348 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:05.506366 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:05.506461 (+ 95us) server_negotiation.cc:407] Connection header received
0805 23:18:05.507359 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:05.507382 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:05.507770 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:05.508176 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:05.509393 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:05.510496 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:05.511756 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:05.512291 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:05.512434 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:05.515618 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:05.515628 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:05.515651 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:05.515691 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:05.517617 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:05.518213 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:05.518219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:05.518224 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:05.518424 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:05.518790 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:05.518793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:05.518794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:05.519795 (+ 1001us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:05.520128 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:05.520758 (+ 630us) server_negotiation.cc:299] Negotiation successful
0805 23:18:05.521170 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":175,"threads_started":1}
I20260805 23:18:05.524077 5855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } }
I20260805 23:18:05.524492 5855 raft_consensus.cc:374] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:05.524588 5855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982c8bc7f60741519afeef9c8276d962, State: Initialized, Role: FOLLOWER
I20260805 23:18:05.525111 5855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } }
I20260805 23:18:05.525271 5855 raft_consensus.cc:388] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:05.525352 5855 raft_consensus.cc:482] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:05.525461 5855 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:05.526794 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:18:05.526773 5855 raft_consensus.cc:504] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } }
I20260805 23:18:05.527206 5855 leader_election.cc:302] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 982c8bc7f60741519afeef9c8276d962; no voters:
I20260805 23:18:05.527648 5855 leader_election.cc:288] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:18:05.528065 5860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:05.528324 5860 raft_consensus.cc:686] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [term 1 LEADER]: Becoming Leader. State: Replica: 982c8bc7f60741519afeef9c8276d962, State: Running, Role: LEADER
I20260805 23:18:05.528743 5860 consensus_queue.cc:237] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } }
I20260805 23:18:05.529536 5855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:05.530124 5860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 982c8bc7f60741519afeef9c8276d962. Latest consensus state: current_term: 1 leader_uuid: "982c8bc7f60741519afeef9c8276d962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } } }
I20260805 23:18:05.530294 5860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:05.530687 5862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "982c8bc7f60741519afeef9c8276d962" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c8bc7f60741519afeef9c8276d962" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42005 } } }
I20260805 23:18:05.530885 5862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:05.530758 5866 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:05.534945 5866 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:05.541777 5866 catalog_manager.cc:1347] Generated new cluster ID: 656e9efc2f814b45bb7601395f778c1e
I20260805 23:18:05.541901 5866 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:05.552925 5866 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:18:05.554018 5866 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:05.559893 5866 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 982c8bc7f60741519afeef9c8276d962: Generated new TSK 0
I20260805 23:18:05.560659 5866 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:18:05.583753 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5784
2026-08-05T23:18:05Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1502 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5886](info): set up 2 sockets
Aug 05 23:18:05 dist-test-slave-vkjp krb5kdc[5886](info): commencing operation
krb5kdc: starting...
W20260805 23:18:06.682452 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:18:06 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:18:06.815302 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:06.930192 5900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:06.930488 5900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:06.930531 5900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:06.934621 5900 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:06.934691 5900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:06.934711 5900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:06.934728 5900 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:06.934787 5900 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:06.940191 5900 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:06.941368 5900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:06.942416 5900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:06.951290 5909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:06.951254 5906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:06.951349 5907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:06.951546 5900 server_base.cc:1034] running on GCE node
I20260805 23:18:06.952361 5900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:06.953630 5900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:06.954859 5900 hybrid_clock.cc:648] HybridClock initialized: now 1785971886954821 us; error 55 us; skew 500 ppm
Aug 05 23:18:06 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971886, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:06.958220 5900 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:06.959487 5900 webserver.cc:460] Webserver started at http://127.23.88.126:38955/ using document root <none> and password file <none>
I20260805 23:18:06.960117 5900 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:06.960222 5900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:06.960481 5900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:06.963030 5900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "3a38de6b336b45e597a23a1bf8585015"
format_stamp: "Formatted at 2026-08-05 23:18:06 on dist-test-slave-vkjp"
server_key: "769bcdcac1b020da29de77fa9c1fc14b"
server_key_iv: "a3d86f2e61a3ba94347359d70a273761"
server_key_version: "encryptionkey@0"
I20260805 23:18:06.963748 5900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "3a38de6b336b45e597a23a1bf8585015"
format_stamp: "Formatted at 2026-08-05 23:18:06 on dist-test-slave-vkjp"
server_key: "769bcdcac1b020da29de77fa9c1fc14b"
server_key_iv: "a3d86f2e61a3ba94347359d70a273761"
server_key_version: "encryptionkey@0"
I20260805 23:18:06.967736 5900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:18:06.970264 5917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:06.971776 5900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:06.971930 5900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3a38de6b336b45e597a23a1bf8585015"
format_stamp: "Formatted at 2026-08-05 23:18:06 on dist-test-slave-vkjp"
server_key: "769bcdcac1b020da29de77fa9c1fc14b"
server_key_iv: "a3d86f2e61a3ba94347359d70a273761"
server_key_version: "encryptionkey@0"
I20260805 23:18:06.972056 5900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:06.987049 5900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:06.990337 5900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:06.990564 5900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:06.998875 5900 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34307
I20260805 23:18:06.999207 5972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34307 every 8 connection(s)
I20260805 23:18:06.999984 5900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:07.002583 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 5900
I20260805 23:18:07.002758 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:18:07.003116 23905 external_mini_cluster.cc:1428] Setting key 5cb1e7e0eb9a0af003f45dd0b635eb61
I20260805 23:18:07.003149 5973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:07.010744 5973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap starting.
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:07.013913 5973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:07.014691 5973 log.cc:824] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:07.017069 5973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: No bootstrap required, opened a new log
I20260805 23:18:07.019038 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.004536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59640 (local address 127.23.88.126:34307)
0805 23:18:07.004967 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.004985 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.005032 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:18:07.005964 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.005982 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.006321 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.006633 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:07.007674 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.008780 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.009978 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.010553 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.010737 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.013811 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:07.013817 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:07.013831 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:07.013864 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:07.015850 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.016366 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.016370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.016375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.016510 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.016806 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.016808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.016809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.017244 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:07.017435 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.017722 (+ 287us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.018033 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":161,"threads_started":1}
I20260805 23:18:07.020541 5973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:07.020833 5973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:07.020887 5973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Initialized, Role: FOLLOWER
I20260805 23:18:07.021379 5973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:07.021569 5973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:07.021679 5973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:07.021802 5973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:07.022812 5973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:07.023221 5973 leader_election.cc:302] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a38de6b336b45e597a23a1bf8585015; no voters:
I20260805 23:18:07.023558 5973 leader_election.cc:288] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:18:07.023778 5979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:07.024078 5979 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 LEADER]: Becoming Leader. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Running, Role: LEADER
I20260805 23:18:07.024422 5979 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:07.024897 5973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:07.026278 5981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a38de6b336b45e597a23a1bf8585015. Latest consensus state: current_term: 1 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:07.026289 5980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:07.026444 5980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:07.026445 5981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:07.026894 5990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:07.030495 5990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:07.036944 5990 catalog_manager.cc:1347] Generated new cluster ID: a39bf97b2b6b42709ac32e17019ea0dd
I20260805 23:18:07.037052 5990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:07.055963 5990 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:18:07.057283 5990 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:07.069020 5990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Generated new TSK 0
I20260805 23:18:07.069967 5990 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:18:07.103148 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:07.220151 6001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:07.220482 6001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:07.220578 6001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:07.224833 6001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:07.224932 6001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:07.225021 6001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:18:07.230620 6001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:07.231810 6001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:07.232923 6001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:07.243010 6007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:07.243026 6008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:07.243026 6010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:07.243346 6001 server_base.cc:1034] running on GCE node
I20260805 23:18:07.243781 6001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:07.244418 6001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:07.245590 6001 hybrid_clock.cc:648] HybridClock initialized: now 1785971887245576 us; error 26 us; skew 500 ppm
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:07.254266 6001 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:18:07.255406 6001 webserver.cc:460] Webserver started at http://127.23.88.65:43033/ using document root <none> and password file <none>
I20260805 23:18:07.255968 6001 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:07.256032 6001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:07.256204 6001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:07.258379 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "d8c5dd0eae3656429769ee2b80fd5288"
server_key_iv: "13f82a6793188f82e7613c631cc3165e"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.258939 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "d8c5dd0eae3656429769ee2b80fd5288"
server_key_iv: "13f82a6793188f82e7613c631cc3165e"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.263068 6001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:18:07.265789 6018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.267335 6001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:18:07.267479 6001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "d8c5dd0eae3656429769ee2b80fd5288"
server_key_iv: "13f82a6793188f82e7613c631cc3165e"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.267616 6001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:07.280972 6001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:07.284566 6001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:07.284795 6001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:07.285457 6001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:07.286337 6001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:07.286409 6001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.286487 6001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:07.286530 6001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.296020 6001 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:37607
I20260805 23:18:07.296093 6145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:37607 every 8 connection(s)
I20260805 23:18:07.297005 6001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:18:07.300796 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6001
I20260805 23:18:07.300946 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:18:07.301246 23905 external_mini_cluster.cc:1428] Setting key f2eff724841c7c68bd43c401aad778a2
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:18:07.313479 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.298939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:53279 (local address 127.23.88.126:34307)
0805 23:18:07.299186 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.299190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.300034 (+ 844us) server_negotiation.cc:407] Connection header received
0805 23:18:07.301736 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.301739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.301802 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.301914 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:07.303387 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.303937 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.305618 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.305868 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.305967 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.309062 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:07.309068 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:07.309074 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:07.309115 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:07.310940 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.311697 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.311700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.311702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.311789 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.312279 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.312282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.312284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.312492 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:07.312602 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.313148 (+ 546us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.313293 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260805 23:18:07.314338 6148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.299342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:53279)
0805 23:18:07.299869 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.299914 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.301489 (+ 1575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.302109 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.302118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.302529 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:07.303236 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.303253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.304084 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.304087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.305463 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.305476 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.306027 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.306033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.306271 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.306955 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:07.306982 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:07.308847 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:07.311131 (+ 2284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.311144 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.311158 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.311562 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.311950 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.311956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.311961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.312170 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.312601 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:07.312610 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:07.312893 (+ 283us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.313090 (+ 197us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.313456 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:18:07.316020 6146 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:07.316309 6146 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:07.316886 6146 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
I20260805 23:18:07.318511 5936 ts_manager.cc:194] Registered new tserver with Master: ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607)
I20260805 23:18:07.319777 5936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:53279
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:18:07.335717 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:07.449643 6152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:07.449980 6152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:07.450065 6152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:07.454320 6152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:07.454430 6152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:07.454530 6152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:18:07.460314 6152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:07.461437 6152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:07.462563 6152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:07.472954 6159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:07.473042 6152 server_base.cc:1034] running on GCE node
W20260805 23:18:07.472981 6160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:07.473223 6163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:07.473634 6152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:07.474330 6152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:07.475500 6152 hybrid_clock.cc:648] HybridClock initialized: now 1785971887475468 us; error 43 us; skew 500 ppm
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:07.478462 6152 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:18:07.479494 6152 webserver.cc:460] Webserver started at http://127.23.88.66:42507/ using document root <none> and password file <none>
I20260805 23:18:07.480085 6152 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:07.480180 6152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:07.480422 6152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:07.482554 6152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "753f97edf1784b20bf998d66d3069107"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "923db1be7884725de222bd78c719bd5a"
server_key_iv: "c3448212110c7584c62dba87cc87a0e3"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.483217 6152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "753f97edf1784b20bf998d66d3069107"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "923db1be7884725de222bd78c719bd5a"
server_key_iv: "c3448212110c7584c62dba87cc87a0e3"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.487671 6152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:18:07.490500 6170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.491962 6152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:07.492139 6152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "753f97edf1784b20bf998d66d3069107"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "923db1be7884725de222bd78c719bd5a"
server_key_iv: "c3448212110c7584c62dba87cc87a0e3"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.492272 6152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:07.512156 6152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:07.515981 6152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:07.516208 6152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:07.516841 6152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:07.517714 6152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:07.517786 6152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.517853 6152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:07.517884 6152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.527434 6152 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35599
I20260805 23:18:07.527468 6291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35599 every 8 connection(s)
I20260805 23:18:07.528412 6152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:18:07.533197 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6152
I20260805 23:18:07.533443 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:18:07.533798 23905 external_mini_cluster.cc:1428] Setting key b8179b9452ae5877c8089752ed339770
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:07.543606 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.530477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:58263 (local address 127.23.88.126:34307)
0805 23:18:07.530654 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.530660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.531583 (+ 923us) server_negotiation.cc:407] Connection header received
0805 23:18:07.532977 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.532981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.533057 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.533188 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:07.534729 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.535363 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.536582 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.536833 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.536957 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.539796 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:07.539799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:07.539802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:07.539827 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:07.541339 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.541901 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.541905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.541906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.541997 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.542362 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.542365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.542367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.542568 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:07.542675 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.543289 (+ 614us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.543433 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:18:07.544586 6297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.530906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:58263)
0805 23:18:07.531429 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.531474 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.532731 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.533466 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.533473 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.533968 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:07.534564 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.534584 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.535501 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.535506 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.536444 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.536454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.537007 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.537011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.537187 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.537984 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:07.538002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:07.539594 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:07.541471 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.541477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.541494 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.541791 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.542094 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.542098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.542100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.542250 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.542691 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:07.542699 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:07.542974 (+ 275us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.543219 (+ 245us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.543598 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":153,"threads_started":1}
I20260805 23:18:07.546080 6293 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:07.546353 6293 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:07.546926 6293 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
I20260805 23:18:07.548146 5936 ts_manager.cc:194] Registered new tserver with Master: 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599)
I20260805 23:18:07.548743 5936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:58263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:18:07.566255 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:07.683821 6301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:07.684093 6301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:07.684139 6301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:07.688287 6301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:07.688366 6301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:07.688452 6301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:18:07.694665 6301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:07.696003 6301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:07.697196 6301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:07.707553 6307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:07.707577 6309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:07.707666 6301 server_base.cc:1034] running on GCE node
W20260805 23:18:07.707587 6306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:07.708433 6301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:07.709276 6301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:07.710513 6301 hybrid_clock.cc:648] HybridClock initialized: now 1785971887710501 us; error 28 us; skew 500 ppm
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:07.713958 6301 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:18:07.714897 6301 webserver.cc:460] Webserver started at http://127.23.88.67:35333/ using document root <none> and password file <none>
I20260805 23:18:07.715561 6301 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:07.715631 6301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:07.715852 6301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:07.717967 6301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "2f74b476538f44cb8a0ee415afbb05b3"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "ecbcb3d6a89f016686c832fa4fe9d5b5"
server_key_iv: "f875a9e76a19e6016cd8fb158f17121c"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.718552 6301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "2f74b476538f44cb8a0ee415afbb05b3"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "ecbcb3d6a89f016686c832fa4fe9d5b5"
server_key_iv: "f875a9e76a19e6016cd8fb158f17121c"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.722674 6301 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:18:07.726293 6316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.728473 6301 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:18:07.728610 6301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "2f74b476538f44cb8a0ee415afbb05b3"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "ecbcb3d6a89f016686c832fa4fe9d5b5"
server_key_iv: "f875a9e76a19e6016cd8fb158f17121c"
server_key_version: "encryptionkey@0"
I20260805 23:18:07.728788 6301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:07.740643 6301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:07.743942 6301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:07.744164 6301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:07.744783 6301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:07.745652 6301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:07.745724 6301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.745792 6301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:07.745838 6301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:07.755334 6301 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41945
I20260805 23:18:07.755352 6437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41945 every 8 connection(s)
I20260805 23:18:07.756330 6301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:18:07.763819 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6301
I20260805 23:18:07.764003 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:18:07.764371 23905 external_mini_cluster.cc:1428] Setting key c69699fc82b52b4cace218d065c3ff9f
Aug 05 23:18:07 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971887, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:07.771628 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.758211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:58513 (local address 127.23.88.126:34307)
0805 23:18:07.758372 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.758377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.759336 (+ 959us) server_negotiation.cc:407] Connection header received
0805 23:18:07.760841 (+ 1505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.760844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.760897 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.760982 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:07.762240 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.762794 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.764234 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.764579 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.764707 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.767888 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:07.767891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:07.767894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:07.767924 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:07.769354 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.769917 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.769920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.769921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.770016 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.770411 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.770413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.770414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.770612 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:07.770688 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.771346 (+ 658us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.771477 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:18:07.772637 6440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.758638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:58513)
0805 23:18:07.759151 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.759213 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.760594 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.761169 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.761177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.761582 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:07.762084 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.762099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.762966 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.762969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.764052 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.764067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.764826 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.764831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.765069 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.765946 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:07.765970 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:07.767706 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:07.769482 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.769487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.769502 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.769803 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.770131 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:07.770133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:07.770135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:07.770312 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:07.770704 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:07.770710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:07.771021 (+ 311us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.771278 (+ 257us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.771692 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260805 23:18:07.773904 6438 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:07.774133 6438 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:07.774618 6438 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
I20260805 23:18:07.775851 5936 ts_manager.cc:194] Registered new tserver with Master: 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945)
I20260805 23:18:07.776530 5936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:58513
I20260805 23:18:07.778738 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:18:07.788780 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.780979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.23.88.126:34307)
0805 23:18:07.781137 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.781142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.781330 (+ 188us) server_negotiation.cc:407] Connection header received
0805 23:18:07.781481 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.781485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.781531 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.781607 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:07.782360 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.782880 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.783770 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.783992 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.784088 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.785487 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:07.785490 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:07.785492 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:07.785518 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:07.787067 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.787535 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.787538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.787539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.787623 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:07.787865 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:07.787867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:07.787868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:07.788123 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:07.788238 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.788441 (+ 203us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.788558 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:18:07.792198 5936 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:18:07.795375 5936 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:18:07.811700 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.804946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.23.88.65:37607)
0805 23:18:07.805793 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.805808 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.805833 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:18:07.805915 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.805920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.806137 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.806291 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:07.806879 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.808122 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.809919 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.810708 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.810832 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.810987 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.811414 (+ 427us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.811519 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":111,"threads_started":1}
I20260805 23:18:07.811753 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.805494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.0.0.1:55360)
0805 23:18:07.806482 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.806497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.806587 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.807147 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.807150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.807169 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:07.807261 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.807267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.808563 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.808567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.809657 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.809667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.810844 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.810848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.811101 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.811240 (+ 139us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.811362 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.811561 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":130,"thread_start_us":98,"threads_started":1}
I20260805 23:18:07.811753 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.804761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.0.0.1:39634)
0805 23:18:07.805295 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.805347 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.805487 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.806505 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.806512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.806551 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:07.806739 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.806746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.808260 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.808263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.809781 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.809793 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.810844 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.810848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.811064 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.811240 (+ 176us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.811362 (+ 122us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.811561 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":386,"spinlock_wait_cycles":7680,"thread_start_us":103,"threads_started":1}
I20260805 23:18:07.812490 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.805836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.23.88.66:35599)
0805 23:18:07.806288 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.806300 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.806694 (+ 394us) server_negotiation.cc:407] Connection header received
0805 23:18:07.806760 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.806771 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.806962 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.807130 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:07.807386 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.808436 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.809817 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.810708 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.810832 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.810987 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.812220 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.812326 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":122,"threads_started":1}
I20260805 23:18:07.815208 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.804973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:41945 (local address 127.0.0.1:49304)
0805 23:18:07.805338 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:07.805356 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:07.805505 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:07.806505 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:07.806512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:07.806549 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:07.806739 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.806746 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.808104 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.808108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.809789 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:07.809794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.812314 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.812319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:07.812417 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.812441 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:18:07.812519 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:18:07.815026 (+ 2507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":122,"spinlock_wait_cycles":5120,"thread_start_us":98,"threads_started":1}
I20260805 23:18:07.815716 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:07.805591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49304 (local address 127.23.88.67:41945)
0805 23:18:07.806058 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:07.806074 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:07.806094 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:18:07.806163 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:07.806168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:07.806329 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:07.806900 (+ 571us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:07.807038 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.807970 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.811067 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:07.811925 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:07.812039 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:07.812192 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:07.815453 (+ 3261us) server_negotiation.cc:299] Negotiation successful
0805 23:18:07.815572 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":154,"threads_started":1}
I20260805 23:18:07.815289 6221 tablet_service.cc:1467] Processing CreateTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 (DEFAULT_TABLE table=test-table [id=8b2cbdf1a0bf4d5ca5755ffeb64b2dcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:07.815277 6074 tablet_service.cc:1467] Processing CreateTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 (DEFAULT_TABLE table=test-table [id=8b2cbdf1a0bf4d5ca5755ffeb64b2dcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:07.816490 6074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9aab2d9e366432a9f1c50c6aa657b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:07.816490 6221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9aab2d9e366432a9f1c50c6aa657b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:07.818987 6367 tablet_service.cc:1467] Processing CreateTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 (DEFAULT_TABLE table=test-table [id=8b2cbdf1a0bf4d5ca5755ffeb64b2dcc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:07.820278 6367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9aab2d9e366432a9f1c50c6aa657b12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:07.823418 6456 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Bootstrap starting.
I20260805 23:18:07.826301 6456 tablet_bootstrap.cc:654] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:07.827428 6456 log.cc:824] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:07.827714 6458 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Bootstrap starting.
I20260805 23:18:07.828197 6457 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap starting.
I20260805 23:18:07.829561 6456 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: No bootstrap required, opened a new log
I20260805 23:18:07.829805 6456 ts_tablet_manager.cc:1397] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:18:07.830561 6458 tablet_bootstrap.cc:654] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:07.831537 6458 log.cc:824] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:07.832257 6457 tablet_bootstrap.cc:654] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:07.833564 6457 log.cc:824] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:07.833747 6456 raft_consensus.cc:348] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.834038 6456 raft_consensus.cc:374] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:07.834100 6456 raft_consensus.cc:729] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753f97edf1784b20bf998d66d3069107, State: Initialized, Role: FOLLOWER
I20260805 23:18:07.834214 6458 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: No bootstrap required, opened a new log
I20260805 23:18:07.834431 6458 ts_tablet_manager.cc:1397] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:18:07.834623 6456 consensus_queue.cc:260] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.835717 6456 ts_tablet_manager.cc:1428] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:18:07.835863 6457 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: No bootstrap required, opened a new log
I20260805 23:18:07.836063 6457 ts_tablet_manager.cc:1397] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260805 23:18:07.836133 6293 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
I20260805 23:18:07.838665 6458 raft_consensus.cc:348] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.838979 6458 raft_consensus.cc:374] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:07.839039 6458 raft_consensus.cc:729] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f74b476538f44cb8a0ee415afbb05b3, State: Initialized, Role: FOLLOWER
I20260805 23:18:07.839546 6458 consensus_queue.cc:260] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.840319 6438 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
I20260805 23:18:07.840689 6458 ts_tablet_manager.cc:1428] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:18:07.840554 6457 raft_consensus.cc:348] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.840812 6457 raft_consensus.cc:374] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:07.840873 6457 raft_consensus.cc:729] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Initialized, Role: FOLLOWER
I20260805 23:18:07.841352 6457 consensus_queue.cc:260] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:07.842082 6146 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
I20260805 23:18:07.842427 6457 ts_tablet_manager.cc:1428] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260805 23:18:08.007395 6439 tablet.cc:2369] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:18:08.029712 6294 tablet.cc:2369] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:18:08.048436 6147 tablet.cc:2369] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:18:08.122653 6465 raft_consensus.cc:482] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:08.122929 6465 raft_consensus.cc:504] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:08.124370 6465 leader_election.cc:288] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945), 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599)
I20260805 23:18:08.128822 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.124888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:45803 (local address 127.23.88.67:41945)
0805 23:18:08.125119 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.125125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.125262 (+ 137us) server_negotiation.cc:407] Connection header received
0805 23:18:08.125482 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.125484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.125551 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.125642 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:08.126019 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.126533 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.127813 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.128220 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.128354 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.128410 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.128594 (+ 184us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.128671 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:18:08.128839 6455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.124885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:50795 (local address 127.23.88.66:35599)
0805 23:18:08.125281 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.125288 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.125563 (+ 275us) server_negotiation.cc:407] Connection header received
0805 23:18:08.125678 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.125681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.125734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.125837 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:08.126134 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.126742 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.127862 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.128257 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.128393 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.128449 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.128625 (+ 176us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.128701 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308}
I20260805 23:18:08.128850 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.124715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:41945 (local address 127.23.88.65:45803)
0805 23:18:08.125142 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.125161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:08.125369 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:08.125683 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:08.125686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:08.125704 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:08.125900 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.125906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.126651 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.126655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.127672 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.127682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.128354 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.128358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.128459 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.128482 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:18:08.128578 (+ 96us) client_negotiation.cc:240] Negotiation successful
0805 23:18:08.128654 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":2,"thread_start_us":148,"threads_started":1}
I20260805 23:18:08.128856 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.125173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.23.88.65:50795)
0805 23:18:08.125454 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.125469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:08.125569 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:08.125849 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:08.125852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:08.125870 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:08.126017 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.126022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.126884 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.126887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.127726 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.127736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.128384 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.128388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.128500 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.128518 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:18:08.128607 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 23:18:08.128687 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":21,"thread_start_us":126,"threads_started":1}
I20260805 23:18:08.129626 6243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9aab2d9e366432a9f1c50c6aa657b12" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753f97edf1784b20bf998d66d3069107" is_pre_election: true
I20260805 23:18:08.129626 6389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9aab2d9e366432a9f1c50c6aa657b12" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74b476538f44cb8a0ee415afbb05b3" is_pre_election: true
I20260805 23:18:08.129966 6243 raft_consensus.cc:2454] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 0.
I20260805 23:18:08.129966 6389 raft_consensus.cc:2454] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 0.
I20260805 23:18:08.130499 6019 leader_election.cc:302] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2f74b476538f44cb8a0ee415afbb05b3, 753f97edf1784b20bf998d66d3069107, ab15f3c05d2c4069ab8f2d8ba0e2843c; no voters:
I20260805 23:18:08.130801 6465 raft_consensus.cc:2781] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:18:08.130888 6465 raft_consensus.cc:482] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:08.130919 6465 raft_consensus.cc:3037] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:08.132088 6465 raft_consensus.cc:504] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:08.132548 6465 leader_election.cc:288] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 1 election: Requested vote from peers 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945), 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599)
I20260805 23:18:08.132916 6389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9aab2d9e366432a9f1c50c6aa657b12" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74b476538f44cb8a0ee415afbb05b3"
I20260805 23:18:08.132933 6243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9aab2d9e366432a9f1c50c6aa657b12" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753f97edf1784b20bf998d66d3069107"
I20260805 23:18:08.133083 6243 raft_consensus.cc:3037] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:08.133090 6389 raft_consensus.cc:3037] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:08.134218 6243 raft_consensus.cc:2454] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 1.
I20260805 23:18:08.134265 6389 raft_consensus.cc:2454] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 1.
I20260805 23:18:08.134548 6020 leader_election.cc:302] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753f97edf1784b20bf998d66d3069107, ab15f3c05d2c4069ab8f2d8ba0e2843c; no voters:
I20260805 23:18:08.134756 6465 raft_consensus.cc:2781] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:08.134991 6465 raft_consensus.cc:686] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 LEADER]: Becoming Leader. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Running, Role: LEADER
I20260805 23:18:08.135367 6465 consensus_queue.cc:237] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:08.138558 5936 catalog_manager.cc:5495] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c reported cstate change: term changed from 0 to 1, leader changed from <none> to ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65). New cstate: current_term: 1 leader_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } health_report { overall_health: UNKNOWN } } }
I20260805 23:18:08.177597 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.173901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.23.88.65:37607)
0805 23:18:08.174038 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.174042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.174072 (+ 30us) server_negotiation.cc:407] Connection header received
0805 23:18:08.174222 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.174224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.174283 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.174370 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:08.174631 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.175189 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.176081 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.176303 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.176399 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.176762 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:08.177234 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:18:08.177391 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.177423 (+ 32us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.177478 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:18:08.184108 6389 raft_consensus.cc:1261] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Refusing update from remote peer ab15f3c05d2c4069ab8f2d8ba0e2843c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:08.184108 6243 raft_consensus.cc:1261] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Refusing update from remote peer ab15f3c05d2c4069ab8f2d8ba0e2843c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:08.184906 6465 consensus_queue.cc:1035] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [LEADER]: Connected to new peer: Peer: permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:08.185128 6468 consensus_queue.cc:1035] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:08.192091 6471 mvcc.cc:205] Tried to move back new op lower bound from 7315340853994270720 to 7315340853804924928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340853994270720}]
I20260805 23:18:08.194587 6472 mvcc.cc:205] Tried to move back new op lower bound from 7315340853994270720 to 7315340853804924928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340853994270720}]
I20260805 23:18:08.194764 6473 mvcc.cc:205] Tried to move back new op lower bound from 7315340853994270720 to 7315340853804924928. Current Snapshot: MvccSnapshot[applied={T|T < 7315340853994270720}]
I20260805 23:18:08.201188 5935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:08.201397 5935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:08.203835 5935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Sending DeleteTablet for 3 replicas of tablet e9aab2d9e366432a9f1c50c6aa657b12
I20260805 23:18:08.204679 6367 tablet_service.cc:1514] Processing DeleteTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:49304
I20260805 23:18:08.204797 6074 tablet_service.cc:1514] Processing DeleteTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39634
I20260805 23:18:08.204885 6221 tablet_service.cc:1514] Processing DeleteTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55360
I20260805 23:18:08.205312 6482 tablet_replica.cc:330] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: stopping tablet replica
I20260805 23:18:08.205392 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6001
I20260805 23:18:08.206388 6481 tablet_replica.cc:330] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: stopping tablet replica
I20260805 23:18:08.206851 6481 raft_consensus.cc:2229] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:08.207257 6481 raft_consensus.cc:2258] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:08.209038 6481 ts_tablet_manager.cc:1905] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:08.211347 6483 tablet_replica.cc:330] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: stopping tablet replica
I20260805 23:18:08.211722 6483 raft_consensus.cc:2229] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:08.212131 6483 raft_consensus.cc:2258] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:08.212699 6481 ts_tablet_manager.cc:1918] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:08.212812 6481 log.cc:1196] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/e9aab2d9e366432a9f1c50c6aa657b12
I20260805 23:18:08.213166 6481 ts_tablet_manager.cc:1939] T e9aab2d9e366432a9f1c50c6aa657b12 P 2f74b476538f44cb8a0ee415afbb05b3: Deleting consensus metadata
I20260805 23:18:08.214269 6483 ts_tablet_manager.cc:1905] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:08.214855 5918 catalog_manager.cc:4841] TS 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945): tablet e9aab2d9e366432a9f1c50c6aa657b12 (table test-table [id=8b2cbdf1a0bf4d5ca5755ffeb64b2dcc]) successfully deleted
W20260805 23:18:08.215410 5919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:37607 (error 108)
I20260805 23:18:08.216323 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6152
I20260805 23:18:08.216648 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.216266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.0.0.1:39640)
0805 23:18:08.216458 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.216542 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260805 23:18:08.218981 5919 catalog_manager.cc:4584] TS ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e9aab2d9e366432a9f1c50c6aa657b12: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37607: connect: Connection refused (error 111)
I20260805 23:18:08.223146 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6301
I20260805 23:18:08.225904 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.224545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.0.0.1:55364)
0805 23:18:08.225723 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.225792 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1076}
I20260805 23:18:08.232084 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 5900
I20260805 23:18:08.239297 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--webserver_interface=127.23.88.126
--webserver_port=38955
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:08.357875 6485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:08.358209 6485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:08.358256 6485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:08.362596 6485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:08.362687 6485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:08.362707 6485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:08.362725 6485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:08.362740 6485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:08.368628 6485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=38955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:08.369927 6485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:08.371114 6485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:08.387686 6494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:08.387686 6491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:08.387796 6485 server_base.cc:1034] running on GCE node
W20260805 23:18:08.387686 6492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:08.388976 6485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:08.390260 6485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:08.391445 6485 hybrid_clock.cc:648] HybridClock initialized: now 1785971888391431 us; error 33 us; skew 500 ppm
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:08.394671 6485 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:08.395780 6485 webserver.cc:460] Webserver started at http://127.23.88.126:38955/ using document root <none> and password file <none>
I20260805 23:18:08.396430 6485 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:08.396512 6485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:08.400732 6485 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:18:08.403232 6501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:08.404642 6485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:08.404807 6485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3a38de6b336b45e597a23a1bf8585015"
format_stamp: "Formatted at 2026-08-05 23:18:06 on dist-test-slave-vkjp"
server_key: "769bcdcac1b020da29de77fa9c1fc14b"
server_key_iv: "a3d86f2e61a3ba94347359d70a273761"
server_key_version: "encryptionkey@0"
I20260805 23:18:08.405480 6485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:08.425279 6485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:08.428776 6485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:08.428959 6485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:08.436618 6485 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34307
I20260805 23:18:08.436968 6560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34307 every 8 connection(s)
I20260805 23:18:08.437880 6485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:08.442637 6561 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:18:08.444584 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap starting.
I20260805 23:18:08.446980 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6485
I20260805 23:18:08.447552 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:37607
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=43033
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:08.465354 6561 log.cc:824] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:08.474776 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:08.475282 6561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap complete.
I20260805 23:18:08.479817 6561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:08.480312 6561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Initialized, Role: FOLLOWER
I20260805 23:18:08.480855 6561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:08.480978 6561 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:08.481097 6561 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:08.481212 6561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:18:08.483579 6561 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:08.483961 6561 leader_election.cc:302] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a38de6b336b45e597a23a1bf8585015; no voters:
I20260805 23:18:08.484344 6561 leader_election.cc:288] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:18:08.484434 6565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:18:08.484655 6565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 LEADER]: Becoming Leader. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Running, Role: LEADER
I20260805 23:18:08.484985 6565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:08.485744 6561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:08.486588 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a38de6b336b45e597a23a1bf8585015. Latest consensus state: current_term: 2 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:08.486629 6566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:08.486727 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:08.486749 6566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:08.491768 6581 catalog_manager.cc:1259] Loaded cluster ID: a39bf97b2b6b42709ac32e17019ea0dd
I20260805 23:18:08.491863 6581 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: loading cluster ID for follower catalog manager: success
W20260805 23:18:08.492868 6581 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260805 23:18:08.492959 6581 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260805 23:18:08.493044 6573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:08.495571 6573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:08.496068 6573 catalog_manager.cc:1259] Loaded cluster ID: a39bf97b2b6b42709ac32e17019ea0dd
I20260805 23:18:08.496130 6573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:08.497586 6573 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260805 23:18:08.502257 6573 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260805 23:18:08.508572 6573 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:08.509619 6573 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Loaded TSK: 0
I20260805 23:18:08.510991 6573 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:18:08.578094 6563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:08.578385 6563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:08.578440 6563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:08.583081 6563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:08.583206 6563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:08.583310 6563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:18:08.591475 6563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:37607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=43033
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:08.593122 6563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:08.594580 6563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:08.607561 6589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:08.607867 6590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:08.607581 6592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:08.608471 6563 server_base.cc:1034] running on GCE node
I20260805 23:18:08.609045 6563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:08.609951 6563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:08.611157 6563 hybrid_clock.cc:648] HybridClock initialized: now 1785971888611115 us; error 59 us; skew 500 ppm
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:08.614593 6563 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:18:08.615924 6563 webserver.cc:460] Webserver started at http://127.23.88.65:43033/ using document root <none> and password file <none>
I20260805 23:18:08.616676 6563 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:08.616806 6563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:08.622319 6563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260805 23:18:08.624917 6601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:08.626456 6563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:08.626616 6563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "d8c5dd0eae3656429769ee2b80fd5288"
server_key_iv: "13f82a6793188f82e7613c631cc3165e"
server_key_version: "encryptionkey@0"
I20260805 23:18:08.627256 6563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:08.642234 6563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:08.645908 6563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:08.646129 6563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:08.646771 6563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:08.648079 6609 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:18:08.652616 6563 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:18:08.652693 6563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260805 23:18:08.652750 6563 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:18:08.654569 6563 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:18:08.654639 6563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:18:08.654903 6609 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap starting.
I20260805 23:18:08.666313 6563 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:37607
I20260805 23:18:08.666397 6722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:37607 every 8 connection(s)
I20260805 23:18:08.667438 6563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:18:08.676519 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6563
I20260805 23:18:08.677066 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:35599
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=42507
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:08.677275 6609 log.cc:824] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:08.686661 6726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.669746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:59757)
0805 23:18:08.670298 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.670341 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:08.671241 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:08.672379 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:08.672387 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:08.672847 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:08.673305 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.673321 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.674798 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.674801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.675749 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.675761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.676518 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.676523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.676808 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.677522 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:08.677539 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:08.679597 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:08.682872 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.682878 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.682891 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.683303 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.683787 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.683790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.683792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.683976 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.684831 (+ 855us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:08.684838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:08.685125 (+ 287us) client_negotiation.cc:769] Sending connection context
0805 23:18:08.685323 (+ 198us) client_negotiation.cc:240] Negotiation successful
0805 23:18:08.685670 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":164,"threads_started":1}
I20260805 23:18:08.686918 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.669629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:59757 (local address 127.23.88.126:34307)
0805 23:18:08.670001 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.670018 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.670471 (+ 453us) server_negotiation.cc:407] Connection header received
0805 23:18:08.671664 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.671674 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.671986 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.672322 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:08.673498 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.674617 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.675881 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.676349 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.676541 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.679898 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:08.679904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:08.679917 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:08.679952 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:08.682711 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.683473 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.683482 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.683489 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.683669 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.684101 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.684107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.684111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.684683 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:08.684857 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.685519 (+ 662us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.685921 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":127,"threads_started":1}
I20260805 23:18:08.687222 6609 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:08.687690 6609 tablet_bootstrap.cc:492] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap complete.
I20260805 23:18:08.689059 6609 ts_tablet_manager.cc:1397] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.012s
I20260805 23:18:08.689777 6723 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:08.690059 6723 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:08.690763 6723 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
I20260805 23:18:08.691828 6609 raft_consensus.cc:348] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:08.692281 6609 raft_consensus.cc:729] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Initialized, Role: FOLLOWER
I20260805 23:18:08.692836 6609 consensus_queue.cc:260] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } }
I20260805 23:18:08.693842 6609 ts_tablet_manager.cc:1428] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:18:08.693964 6521 ts_manager.cc:194] Registered new tserver with Master: ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607)
I20260805 23:18:08.698180 6521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:59757
I20260805 23:18:08.700870 6723 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:18:08.709785 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.697599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.0.0.1:39644)
0805 23:18:08.698425 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.698463 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:08.698640 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:08.699188 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:08.699194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:08.699667 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:08.699817 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.699826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.701326 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.701329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.702078 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.702088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.702712 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.702715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.702848 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.703502 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:08.703514 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:08.705384 (+ 1870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:08.707685 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.707697 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.707703 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.707985 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.708331 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.708372 (+ 41us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.708374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.708479 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.709012 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:08.709019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:08.709171 (+ 152us) client_negotiation.cc:769] Sending connection context
0805 23:18:08.709322 (+ 151us) client_negotiation.cc:240] Negotiation successful
0805 23:18:08.709592 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":137,"threads_started":1}
I20260805 23:18:08.709785 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.697638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.23.88.65:37607)
0805 23:18:08.698158 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.698169 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.698571 (+ 402us) server_negotiation.cc:407] Connection header received
0805 23:18:08.698821 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.698826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.698998 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.699187 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:08.699965 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.701190 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.702208 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.702573 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.702694 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.705584 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:08.705595 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:08.705608 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:08.705635 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:08.707546 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.708134 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.708140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.708141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.708227 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.708580 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.708584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.708586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.708882 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:08.709054 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.709408 (+ 354us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.709592 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":134,"threads_started":1}
I20260805 23:18:08.710918 6653 tablet_service.cc:1514] Processing DeleteTablet for tablet e9aab2d9e366432a9f1c50c6aa657b12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:08 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39644
I20260805 23:18:08.712013 6732 tablet_replica.cc:330] stopping tablet replica
I20260805 23:18:08.712335 6732 raft_consensus.cc:2229] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:08.712597 6732 raft_consensus.cc:2258] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:08.713698 6732 ts_tablet_manager.cc:1905] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:08.718040 6732 ts_tablet_manager.cc:1918] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:08.718178 6732 log.cc:1196] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/e9aab2d9e366432a9f1c50c6aa657b12
I20260805 23:18:08.718480 6732 ts_tablet_manager.cc:1939] T e9aab2d9e366432a9f1c50c6aa657b12 P ab15f3c05d2c4069ab8f2d8ba0e2843c: Deleting consensus metadata
I20260805 23:18:08.720374 6503 catalog_manager.cc:4841] TS ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607): tablet e9aab2d9e366432a9f1c50c6aa657b12 (table test-table [id=8b2cbdf1a0bf4d5ca5755ffeb64b2dcc]) successfully deleted
W20260805 23:18:08.806274 6728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:08.806566 6728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:08.806643 6728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:08.811203 6728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:08.811285 6728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:08.811376 6728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:18:08.817569 6728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:35599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=42507
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:08.818795 6728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:08.819988 6728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:08.830379 6738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:08.830379 6741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:08.830405 6737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:08.830508 6728 server_base.cc:1034] running on GCE node
I20260805 23:18:08.831281 6728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:08.832077 6728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:08.833300 6728 hybrid_clock.cc:648] HybridClock initialized: now 1785971888833280 us; error 34 us; skew 500 ppm
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:08.836575 6728 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:18:08.837633 6728 webserver.cc:460] Webserver started at http://127.23.88.66:42507/ using document root <none> and password file <none>
I20260805 23:18:08.838228 6728 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:08.838330 6728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:08.842531 6728 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:08.845047 6748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:08.846482 6728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:08.846640 6728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "753f97edf1784b20bf998d66d3069107"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "923db1be7884725de222bd78c719bd5a"
server_key_iv: "c3448212110c7584c62dba87cc87a0e3"
server_key_version: "encryptionkey@0"
I20260805 23:18:08.847324 6728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:08.873247 6728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:08.876909 6728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:08.877154 6728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:08.877827 6728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:08.879168 6755 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260805 23:18:08.883528 6755 ts_tablet_manager.cc:1852] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260805 23:18:08.883989 6755 ts_tablet_manager.cc:1905] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:08.889068 6755 ts_tablet_manager.cc:1918] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:08.889209 6755 log.cc:1196] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/e9aab2d9e366432a9f1c50c6aa657b12
I20260805 23:18:08.889638 6755 ts_tablet_manager.cc:1939] T e9aab2d9e366432a9f1c50c6aa657b12 P 753f97edf1784b20bf998d66d3069107: Deleting consensus metadata
I20260805 23:18:08.890044 6728 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260805 23:18:08.890154 6728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.001s sys 0.000s
I20260805 23:18:08.890275 6728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:08.890342 6728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:08.900552 6728 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35599
I20260805 23:18:08.900583 6872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35599 every 8 connection(s)
I20260805 23:18:08.901546 6728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:18:08.907303 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6728
I20260805 23:18:08.907953 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:41945
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=35333
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:18:08 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971888, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:08.922161 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.903514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:35735 (local address 127.23.88.126:34307)
0805 23:18:08.903659 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:08.903662 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:08.904571 (+ 909us) server_negotiation.cc:407] Connection header received
0805 23:18:08.905606 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:08.905609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:08.905678 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:08.905784 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:08.907024 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.907775 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.912255 (+ 4480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.912770 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.912938 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.916819 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:08.916828 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:08.916832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:08.916881 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:08.919048 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.919854 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.919860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.919864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.919970 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:08.920475 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:08.920480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:08.920483 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:08.920799 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:08.920932 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:08.921715 (+ 783us) server_negotiation.cc:299] Negotiation successful
0805 23:18:08.921906 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260805 23:18:08.923386 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:08.903873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:35735)
0805 23:18:08.904396 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:08.904446 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:08.905371 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:08.905975 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:08.905982 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:08.906347 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:08.906864 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.906878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.907965 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.907970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.909610 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:08.909626 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:08.913007 (+ 3381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:08.913012 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:08.913315 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:08.914270 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:08.914295 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:08.916510 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:08.919239 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.919250 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.919271 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.919722 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.920113 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:08.920119 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:08.920124 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:08.920341 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:08.920945 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:08.920955 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:08.921371 (+ 416us) client_negotiation.cc:769] Sending connection context
0805 23:18:08.921641 (+ 270us) client_negotiation.cc:240] Negotiation successful
0805 23:18:08.922072 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":132,"threads_started":1}
I20260805 23:18:08.924921 6873 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:08.925210 6873 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:08.925889 6873 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
I20260805 23:18:08.927246 6521 ts_manager.cc:194] Registered new tserver with Master: 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599)
I20260805 23:18:08.928033 6521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:35735
W20260805 23:18:09.034624 6876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:09.034924 6876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:09.034974 6876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:09.039551 6876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:09.039621 6876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:09.039711 6876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:18:09.045493 6876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:41945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=35333
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:09.046695 6876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:09.047873 6876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:09.058966 6885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.058995 6883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.059124 6876 server_base.cc:1034] running on GCE node
W20260805 23:18:09.058916 6882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.059859 6876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:09.060840 6876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:09.062093 6876 hybrid_clock.cc:648] HybridClock initialized: now 1785971889062054 us; error 52 us; skew 500 ppm
Aug 05 23:18:09 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:09.066231 6876 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:18:09.067456 6876 webserver.cc:460] Webserver started at http://127.23.88.67:35333/ using document root <none> and password file <none>
I20260805 23:18:09.068048 6876 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:09.068164 6876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:09.072346 6876 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:18:09.074641 6892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.076103 6876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:09.076238 6876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "2f74b476538f44cb8a0ee415afbb05b3"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "ecbcb3d6a89f016686c832fa4fe9d5b5"
server_key_iv: "f875a9e76a19e6016cd8fb158f17121c"
server_key_version: "encryptionkey@0"
I20260805 23:18:09.076828 6876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:09.095820 6876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:09.099362 6876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:09.099578 6876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:09.100268 6876 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:09.101156 6876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:09.101207 6876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.101274 6876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:09.101306 6876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.112677 6876 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41945
I20260805 23:18:09.112697 7018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41945 every 8 connection(s)
I20260805 23:18:09.113783 6876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:18:09.119494 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 6876
Aug 05 23:18:09 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:09.131526 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.120347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59660 (local address 127.23.88.126:34307)
0805 23:18:09.120667 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.120673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.121019 (+ 346us) server_negotiation.cc:407] Connection header received
0805 23:18:09.121159 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.121162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.121211 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.121302 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:09.122368 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.123054 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.124990 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.125338 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.125463 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.127134 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:09.127138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:09.127141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:09.127190 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:09.128989 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:09.129534 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:09.129540 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:09.129543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:09.129660 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:09.130000 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:09.130003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:09.130005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:09.130218 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:09.131031 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.131138 (+ 107us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.131278 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":118,"threads_started":1}
I20260805 23:18:09.131572 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.116136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:35491 (local address 127.23.88.126:34307)
0805 23:18:09.116286 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.116290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.117569 (+ 1279us) server_negotiation.cc:407] Connection header received
0805 23:18:09.119031 (+ 1462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.119035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.119115 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.119198 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:09.120729 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.121463 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.122801 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.123278 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.123416 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.126769 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:09.126775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:09.126780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:09.126814 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:09.129114 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:09.129765 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:09.129769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:09.129771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:09.129861 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:09.130253 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:09.130257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:09.130262 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:09.130463 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:09.130569 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.131301 (+ 732us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.131428 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:18:09.132805 7022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.116801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:35491)
0805 23:18:09.117401 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.117455 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.118803 (+ 1348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.119374 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:09.119381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:09.119826 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:09.120571 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.120590 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.121593 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.121603 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.122665 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.122676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.123534 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.123538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.123694 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.124505 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:09.124522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:09.126538 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:09.129248 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:09.129256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:09.129275 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:09.129657 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:09.129985 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:09.129988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:09.129990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:09.130149 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:09.130585 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:09.130593 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:09.130965 (+ 372us) client_negotiation.cc:769] Sending connection context
0805 23:18:09.131243 (+ 278us) client_negotiation.cc:240] Negotiation successful
0805 23:18:09.131679 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":195,"threads_started":1}
I20260805 23:18:09.133934 7019 heartbeater.cc:347] Connected to a master server at 127.23.88.126:34307
I20260805 23:18:09.134297 7019 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:09.134842 7019 heartbeater.cc:511] Master 127.23.88.126:34307 requested a full tablet report, sending...
I20260805 23:18:09.135986 6521 ts_manager.cc:194] Registered new tserver with Master: 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945)
I20260805 23:18:09.136519 6521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:35491
I20260805 23:18:09.145480 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:18:09.149879 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.146105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59668 (local address 127.23.88.126:34307)
0805 23:18:09.146216 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.146220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.146479 (+ 259us) server_negotiation.cc:407] Connection header received
0805 23:18:09.146611 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.146613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.146722 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.146812 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:09.147044 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.147644 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.148507 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.148724 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.148800 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.149081 (+ 281us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:09.149490 (+ 409us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:18:09.149579 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.149707 (+ 128us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.149763 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260805 23:18:09.150622 6521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:18:09.154260 6521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:18:09.163650 6653 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb (DEFAULT_TABLE table=test-table [id=453de3048b0245f293bb13bc62bd3be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:09.164366 6653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0c2d3a69b5459db30b823892ca49bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:09.167577 6609 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap starting.
I20260805 23:18:09.168248 6730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.162250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.0.0.1:55372)
0805 23:18:09.162406 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.162423 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.162511 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.163850 (+ 1339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:09.163853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:09.163872 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:09.163993 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.164000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.165722 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.165725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.166634 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.166644 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.167788 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.167792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.167893 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.167956 (+ 63us) client_negotiation.cc:769] Sending connection context
0805 23:18:09.168014 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 23:18:09.168103 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":49}
I20260805 23:18:09.168612 6609 tablet_bootstrap.cc:654] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:09.168937 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.162492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.23.88.66:35599)
0805 23:18:09.163305 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.163318 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.163387 (+ 69us) server_negotiation.cc:407] Connection header received
0805 23:18:09.163454 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.163458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.163664 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.163838 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:09.164126 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.165563 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.166772 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.167641 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.168422 (+ 781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.168604 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.168662 (+ 58us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.168763 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":701,"thread_start_us":155,"threads_started":1}
I20260805 23:18:09.170142 6609 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: No bootstrap required, opened a new log
I20260805 23:18:09.170259 6609 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:09.170689 6609 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.170392 6800 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb (DEFAULT_TABLE table=test-table [id=453de3048b0245f293bb13bc62bd3be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:09.170814 6609 raft_consensus.cc:374] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:09.170872 6609 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.171051 6609 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.171518 6609 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260805 23:18:09.171540 6800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0c2d3a69b5459db30b823892ca49bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:09.173923 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.162615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:41945 (local address 127.0.0.1:49308)
0805 23:18:09.162939 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.162954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.163049 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.164126 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:09.164129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:09.164144 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:09.164219 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.164224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.165669 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.165672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.166899 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.166910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.173423 (+ 6513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.173429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.173541 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.173566 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:18:09.173655 (+ 89us) client_negotiation.cc:240] Negotiation successful
0805 23:18:09.173761 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":40,"thread_start_us":151,"threads_started":1}
I20260805 23:18:09.173931 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.162820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49308 (local address 127.23.88.67:41945)
0805 23:18:09.163305 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.163319 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.163343 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:18:09.163433 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.163438 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.163664 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.164009 (+ 345us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:09.164345 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.165541 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.167052 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.172239 (+ 5187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.172490 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.172632 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.173702 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.173811 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":163,"threads_started":1}
I20260805 23:18:09.177220 6946 tablet_service.cc:1467] Processing CreateTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb (DEFAULT_TABLE table=test-table [id=453de3048b0245f293bb13bc62bd3be4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:09.178048 6755 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Bootstrap starting.
I20260805 23:18:09.178426 6946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0c2d3a69b5459db30b823892ca49bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:09.180256 6755 tablet_bootstrap.cc:654] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:09.181236 6755 log.cc:824] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:09.183056 6755 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: No bootstrap required, opened a new log
I20260805 23:18:09.183311 6755 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:18:09.186332 6755 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.186633 6755 raft_consensus.cc:374] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:09.186693 6755 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753f97edf1784b20bf998d66d3069107, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.187151 7036 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Bootstrap starting.
I20260805 23:18:09.187225 6755 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.187970 6873 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
I20260805 23:18:09.188280 6755 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:18:09.190968 7036 tablet_bootstrap.cc:654] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:09.192109 7036 log.cc:824] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:09.194536 7036 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: No bootstrap required, opened a new log
I20260805 23:18:09.194761 7036 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260805 23:18:09.195406 7037 raft_consensus.cc:482] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:09.195593 7037 raft_consensus.cc:504] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.196710 7037 leader_election.cc:288] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945), ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607)
I20260805 23:18:09.198624 7036 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.198966 7036 raft_consensus.cc:374] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:09.199051 7036 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f74b476538f44cb8a0ee415afbb05b3, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.199548 7036 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.200316 7019 heartbeater.cc:503] Master 127.23.88.126:34307 was elected leader, sending a full tablet report...
I20260805 23:18:09.200723 7036 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260805 23:18:09.201256 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.196907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:41945 (local address 127.23.88.66:42543)
0805 23:18:09.197201 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.197219 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.197312 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.197617 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:09.197620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:09.197639 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:09.197772 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.197778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.198815 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.198819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.199795 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.199806 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.200746 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.200750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.200882 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.200919 (+ 37us) client_negotiation.cc:769] Sending connection context
0805 23:18:09.200977 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 23:18:09.201077 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260805 23:18:09.201850 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.197196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:42543 (local address 127.23.88.67:41945)
0805 23:18:09.197360 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.197364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.197381 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:18:09.197429 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.197432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.197490 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.197875 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:09.197977 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.198679 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.199957 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.200594 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.201583 (+ 989us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.201637 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.201671 (+ 34us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.201738 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260805 23:18:09.202407 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "753f97edf1784b20bf998d66d3069107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74b476538f44cb8a0ee415afbb05b3" is_pre_election: true
I20260805 23:18:09.202747 6967 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753f97edf1784b20bf998d66d3069107 in term 0.
I20260805 23:18:09.202807 6731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.197078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:51121 (local address 127.23.88.65:37607)
0805 23:18:09.197325 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.197330 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.197564 (+ 234us) server_negotiation.cc:407] Connection header received
0805 23:18:09.197712 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.197715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.197784 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.197889 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:09.198157 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.198692 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.201280 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.202049 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.202188 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.202323 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.202611 (+ 288us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.202699 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260805 23:18:09.202813 7039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.196964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.23.88.66:51121)
0805 23:18:09.197476 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.197486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.197599 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.197899 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:09.197902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:09.197918 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:09.198042 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.198048 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.199973 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.199976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.201082 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:09.201093 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.202399 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.202402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:09.202479 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.202498 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:18:09.202592 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:18:09.202666 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":125,"threads_started":1}
I20260805 23:18:09.203337 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "753f97edf1784b20bf998d66d3069107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" is_pre_election: true
I20260805 23:18:09.203622 6674 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 753f97edf1784b20bf998d66d3069107 in term 0.
I20260805 23:18:09.204021 6749 leader_election.cc:302] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f74b476538f44cb8a0ee415afbb05b3, 753f97edf1784b20bf998d66d3069107; no voters:
I20260805 23:18:09.204358 7037 raft_consensus.cc:2781] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:18:09.204455 7037 raft_consensus.cc:482] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:09.204511 7037 raft_consensus.cc:3037] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:09.205608 7037 raft_consensus.cc:504] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.206012 7037 leader_election.cc:288] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [CANDIDATE]: Term 1 election: Requested vote from peers 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945), ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607)
I20260805 23:18:09.206351 6967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "753f97edf1784b20bf998d66d3069107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f74b476538f44cb8a0ee415afbb05b3"
I20260805 23:18:09.206467 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "753f97edf1784b20bf998d66d3069107" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
I20260805 23:18:09.206494 6967 raft_consensus.cc:3037] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:09.206640 6674 raft_consensus.cc:3037] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:09.207695 6967 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753f97edf1784b20bf998d66d3069107 in term 1.
I20260805 23:18:09.207831 6674 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753f97edf1784b20bf998d66d3069107 in term 1.
I20260805 23:18:09.208041 6749 leader_election.cc:302] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f74b476538f44cb8a0ee415afbb05b3, 753f97edf1784b20bf998d66d3069107; no voters:
I20260805 23:18:09.208258 7037 raft_consensus.cc:2781] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:09.208489 7037 raft_consensus.cc:686] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 LEADER]: Becoming Leader. State: Replica: 753f97edf1784b20bf998d66d3069107, State: Running, Role: LEADER
I20260805 23:18:09.208843 7037 consensus_queue.cc:237] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.212384 6520 catalog_manager.cc:5495] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 reported cstate change: term changed from 0 to 1, leader changed from <none> to 753f97edf1784b20bf998d66d3069107 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "753f97edf1784b20bf998d66d3069107" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } health_report { overall_health: UNKNOWN } } }
I20260805 23:18:09.223841 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.220250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.23.88.66:35599)
0805 23:18:09.220413 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:09.220416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:09.220430 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:18:09.220562 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:09.220564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:09.220620 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:09.220712 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:09.220971 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.221470 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.222313 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:09.222535 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:09.222618 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:09.222894 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:09.223439 (+ 545us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:18:09.223534 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:09.223600 (+ 66us) server_negotiation.cc:299] Negotiation successful
0805 23:18:09.223655 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:18:09.230322 6674 raft_consensus.cc:1261] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Refusing update from remote peer 753f97edf1784b20bf998d66d3069107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:09.230327 6967 raft_consensus.cc:1261] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Refusing update from remote peer 753f97edf1784b20bf998d66d3069107: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:09.231107 7037 consensus_queue.cc:1035] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:09.231309 7041 consensus_queue.cc:1035] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:09.238976 7046 mvcc.cc:205] Tried to move back new op lower bound from 7315340858279059456 to 7315340858202517504. Current Snapshot: MvccSnapshot[applied={T|T < 7315340858279059456}]
I20260805 23:18:09.240006 7045 mvcc.cc:205] Tried to move back new op lower bound from 7315340858279059456 to 7315340858202517504. Current Snapshot: MvccSnapshot[applied={T|T < 7315340858279059456}]
I20260805 23:18:09.248011 6520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:09.248188 6520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:09.251227 6520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Sending DeleteTablet for 3 replicas of tablet 9a0c2d3a69b5459db30b823892ca49bb
I20260805 23:18:09.252221 6653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:09 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39644
I20260805 23:18:09.252246 6946 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:09 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:49308
I20260805 23:18:09.252321 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6563
I20260805 23:18:09.253983 6800 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a0c2d3a69b5459db30b823892ca49bb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:09 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55372
I20260805 23:18:09.254770 7055 tablet_replica.cc:330] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: stopping tablet replica
I20260805 23:18:09.255205 7055 raft_consensus.cc:2229] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:09.255589 7055 raft_consensus.cc:2258] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:09.257103 7055 ts_tablet_manager.cc:1905] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:09.259541 7055 ts_tablet_manager.cc:1918] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:09.259682 7055 log.cc:1196] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/9a0c2d3a69b5459db30b823892ca49bb
I20260805 23:18:09.256095 7057 tablet_replica.cc:330] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: stopping tablet replica
I20260805 23:18:09.260031 7055 ts_tablet_manager.cc:1939] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Deleting consensus metadata
I20260805 23:18:09.260406 7057 raft_consensus.cc:2229] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:18:09.260933 7057 raft_consensus.cc:2258] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:09.261791 6502 catalog_manager.cc:4841] TS 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945): tablet 9a0c2d3a69b5459db30b823892ca49bb (table test-table [id=453de3048b0245f293bb13bc62bd3be4]) successfully deleted
W20260805 23:18:09.262993 6503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:37607 (error 108)
I20260805 23:18:09.263286 7057 ts_tablet_manager.cc:1905] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:09.263618 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6728
I20260805 23:18:09.264618 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.264304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.0.0.1:39656)
0805 23:18:09.264470 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.264529 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260805 23:18:09.264809 6503 catalog_manager.cc:4584] TS ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9a0c2d3a69b5459db30b823892ca49bb: Network error: Client connection negotiation failed: client connection to 127.23.88.65:37607: connect: Connection refused (error 111)
I20260805 23:18:09.272058 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6876
I20260805 23:18:09.272305 7032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.271985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.0.0.1:55390)
0805 23:18:09.272130 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.272200 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:35599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260805 23:18:09.278985 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 6485
I20260805 23:18:09.287508 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--webserver_interface=127.23.88.126
--webserver_port=38955
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:09.404927 7059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:09.405277 7059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:09.405324 7059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:09.409688 7059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:09.409775 7059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:09.409796 7059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:09.409814 7059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:09.409829 7059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:09.415663 7059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=38955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:09.416993 7059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:09.418080 7059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:09.427551 7067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.427698 7059 server_base.cc:1034] running on GCE node
W20260805 23:18:09.427556 7066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.427592 7069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.428541 7059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:09.429783 7059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:09.430981 7059 hybrid_clock.cc:648] HybridClock initialized: now 1785971889430965 us; error 31 us; skew 500 ppm
Aug 05 23:18:09 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:09.434091 7059 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:09.435201 7059 webserver.cc:460] Webserver started at http://127.23.88.126:38955/ using document root <none> and password file <none>
I20260805 23:18:09.435809 7059 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:09.435912 7059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:09.440236 7059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:18:09.442613 7077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.443997 7059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:18:09.444166 7059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "3a38de6b336b45e597a23a1bf8585015"
format_stamp: "Formatted at 2026-08-05 23:18:06 on dist-test-slave-vkjp"
server_key: "769bcdcac1b020da29de77fa9c1fc14b"
server_key_iv: "a3d86f2e61a3ba94347359d70a273761"
server_key_version: "encryptionkey@0"
I20260805 23:18:09.444778 7059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:09.457369 7059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:09.461040 7059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:09.461278 7059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:09.468961 7059 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34307
I20260805 23:18:09.469275 7131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34307 every 8 connection(s)
I20260805 23:18:09.470208 7059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:09.474694 7132 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:18:09.475190 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7059
I20260805 23:18:09.475665 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:37607
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=43033
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:09.476712 7132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap starting.
I20260805 23:18:09.495476 7132 log.cc:824] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:09.507635 7132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:09.508114 7132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015: Bootstrap complete.
I20260805 23:18:09.512908 7132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:09.513412 7132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.513970 7132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:09.514137 7132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:09.514238 7132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:09.514384 7132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:18:09.516880 7132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:09.517319 7132 leader_election.cc:302] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a38de6b336b45e597a23a1bf8585015; no voters:
I20260805 23:18:09.517735 7132 leader_election.cc:288] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 23:18:09.517849 7136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 3 FOLLOWER]: Leader election won for term 3
I20260805 23:18:09.518064 7136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [term 3 LEADER]: Becoming Leader. State: Replica: 3a38de6b336b45e597a23a1bf8585015, State: Running, Role: LEADER
I20260805 23:18:09.518396 7136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } }
I20260805 23:18:09.519210 7132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:09.519557 7136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a38de6b336b45e597a23a1bf8585015. Latest consensus state: current_term: 3 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:09.519697 7136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:09.520097 7137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3a38de6b336b45e597a23a1bf8585015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a38de6b336b45e597a23a1bf8585015" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34307 } } }
I20260805 23:18:09.520363 7137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a38de6b336b45e597a23a1bf8585015 [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:09.521757 7140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:09.527650 7140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:09.528362 7140 catalog_manager.cc:1259] Loaded cluster ID: a39bf97b2b6b42709ac32e17019ea0dd
I20260805 23:18:09.528427 7140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260805 23:18:09.529549 7140 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6b54d789ad google::LogMessage::Fail() at ??:0
@ 0x7f6b54d7b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6b54d784de google::LogMessage::Flush() at ??:0
@ 0x7f6b54d79a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6b5f6b7fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6b5f6b8d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6b5f6b5579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6b5f6f9640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6b56fca3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6b56fcac83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6b56fcc3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6b56fbb512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6b585ff6db start_thread at ??:0
@ 0x7f6b5387661f clone at ??:0
W20260805 23:18:09.629864 7134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:09.630154 7134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:09.630208 7134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:09.634507 7134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:09.634588 7134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:09.634651 7134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:18:09.640491 7134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:37607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=43033
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:09.641745 7134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:09.642951 7134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:09.654369 7184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.654390 7181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.654390 7182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.655277 7134 server_base.cc:1034] running on GCE node
I20260805 23:18:09.655778 7134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:09.656606 7134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:09.657796 7134 hybrid_clock.cc:648] HybridClock initialized: now 1785971889657764 us; error 52 us; skew 500 ppm
Aug 05 23:18:09 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:09.660900 7134 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:18:09.661939 7134 webserver.cc:460] Webserver started at http://127.23.88.65:43033/ using document root <none> and password file <none>
I20260805 23:18:09.662549 7134 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:09.662647 7134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:09.666915 7134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:09.669317 7191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.670671 7134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:09.670775 7134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "d8c5dd0eae3656429769ee2b80fd5288"
server_key_iv: "13f82a6793188f82e7613c631cc3165e"
server_key_version: "encryptionkey@0"
I20260805 23:18:09.671451 7134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:09.698668 7134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:09.702664 7134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:09.702929 7134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:09.703679 7134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:09.705049 7198 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:18:09.711009 7134 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:18:09.711166 7134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260805 23:18:09.711302 7134 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:18:09.713229 7134 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:18:09.713317 7134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:18:09.713625 7198 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap starting.
I20260805 23:18:09.725708 7134 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:37607
I20260805 23:18:09.725729 7312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:37607 every 8 connection(s)
I20260805 23:18:09.726874 7134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:18:09.734887 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7134
I20260805 23:18:09.735426 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:35599
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=42507
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:09.736899 7198 log.cc:824] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:09.745235 7198 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:09.745720 7198 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Bootstrap complete.
I20260805 23:18:09.746999 7198 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent bootstrapping tablet: real 0.034s user 0.017s sys 0.012s
I20260805 23:18:09.749722 7198 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.750228 7198 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.750828 7198 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.752005 7198 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
W20260805 23:18:09.765755 7315 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:18:09.729548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:49417)
0805 23:18:09.730222 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.730282 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:09.731313 (+ 1031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:09.764941 (+ 33628us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":122,"threads_started":1}
W20260805 23:18:09.766417 7313 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:34307 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:34307: Client connection negotiation failed: client connection to 127.23.88.126:34307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:18:09.767349 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.766985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:54633)
0805 23:18:09.767204 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.767284 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260805 23:18:09.768106 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.767865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:60553)
0805 23:18:09.768000 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.768055 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260805 23:18:09.768410 7313 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:18:09.865162 7316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:09.865464 7316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:09.865510 7316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:09.869743 7316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:09.869829 7316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:09.869894 7316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:18:09.875682 7316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:35599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=42507
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:09.876794 7316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:09.877920 7316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:09.888021 7324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.888029 7327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:09.888029 7325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:09.888276 7316 server_base.cc:1034] running on GCE node
I20260805 23:18:09.888854 7316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:09.889577 7316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:09.890744 7316 hybrid_clock.cc:648] HybridClock initialized: now 1785971889890709 us; error 51 us; skew 500 ppm
Aug 05 23:18:09 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:09.893795 7316 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:18:09.894793 7316 webserver.cc:460] Webserver started at http://127.23.88.66:42507/ using document root <none> and password file <none>
I20260805 23:18:09.895438 7316 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:09.895509 7316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:09.899567 7316 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260805 23:18:09.902096 7334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:09.903496 7316 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:09.903605 7316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "753f97edf1784b20bf998d66d3069107"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "923db1be7884725de222bd78c719bd5a"
server_key_iv: "c3448212110c7584c62dba87cc87a0e3"
server_key_version: "encryptionkey@0"
I20260805 23:18:09.904224 7316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:09.935690 7316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:09.939399 7316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:09.939630 7316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:09.940304 7316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:09.941532 7341 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:18:09.946254 7316 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:18:09.946353 7316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260805 23:18:09.946457 7316 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:18:09.948349 7316 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:18:09.948432 7316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260805 23:18:09.948796 7341 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Bootstrap starting.
I20260805 23:18:09.954586 7341 log.cc:824] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:09.960165 7316 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35599
I20260805 23:18:09.960237 7460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35599 every 8 connection(s)
I20260805 23:18:09.961124 7316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:18:09.964197 7341 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:09.964716 7341 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Bootstrap complete.
I20260805 23:18:09.964815 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.963039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:45425)
0805 23:18:09.963633 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.963785 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":119,"threads_started":1}
I20260805 23:18:09.964811 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7316
I20260805 23:18:09.965149 7341 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
W20260805 23:18:09.965301 7461 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:34307 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:34307: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
I20260805 23:18:09.965312 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:41945
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:34307
--webserver_port=35333
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:09.966075 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.965740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:42317)
0805 23:18:09.965912 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.965985 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:18:09.966934 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.966659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:59319)
0805 23:18:09.966796 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.966863 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260805 23:18:09.967221 7461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:18:09.968878 7341 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.969667 7341 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753f97edf1784b20bf998d66d3069107, State: Initialized, Role: FOLLOWER
I20260805 23:18:09.970274 7341 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:09.971621 7341 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260805 23:18:09.972045 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.971633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:50705)
0805 23:18:09.971860 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.971963 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260805 23:18:09.972817 7463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:09.972562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:50893)
0805 23:18:09.972690 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:09.972749 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260805 23:18:10.084791 7464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:10.085085 7464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:10.085171 7464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:10.089351 7464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:10.089462 7464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:10.089555 7464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:18:10.095180 7464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:41945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=35333
--tserver_master_addrs=127.23.88.126:34307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:10.096331 7464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:10.097388 7464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:10.108031 7472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:10.108049 7471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:10.108222 7464 server_base.cc:1034] running on GCE node
W20260805 23:18:10.108126 7474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:10.108841 7464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:10.109616 7464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:10.110791 7464 hybrid_clock.cc:648] HybridClock initialized: now 1785971890110755 us; error 48 us; skew 500 ppm
Aug 05 23:18:10 dist-test-slave-vkjp krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971890, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:10.113893 7464 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:18:10.114902 7464 webserver.cc:460] Webserver started at http://127.23.88.67:35333/ using document root <none> and password file <none>
I20260805 23:18:10.115556 7464 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:10.115622 7464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:10.119980 7464 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:18:10.122285 7482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:10.123695 7464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:10.123860 7464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "2f74b476538f44cb8a0ee415afbb05b3"
format_stamp: "Formatted at 2026-08-05 23:18:07 on dist-test-slave-vkjp"
server_key: "ecbcb3d6a89f016686c832fa4fe9d5b5"
server_key_iv: "f875a9e76a19e6016cd8fb158f17121c"
server_key_version: "encryptionkey@0"
I20260805 23:18:10.124465 7464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:10.145750 7464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:10.149204 7464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:10.149434 7464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:10.150089 7464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:10.150954 7464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:10.151026 7464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:10.151093 7464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:10.151199 7464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:10.161213 7464 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:41945
I20260805 23:18:10.161319 7602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:41945 every 8 connection(s)
I20260805 23:18:10.162351 7464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:18:10.163275 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7464
I20260805 23:18:10.166743 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:10.164876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:37907)
0805 23:18:10.165350 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:10.165475 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":118,"threads_started":1}
W20260805 23:18:10.167291 7603 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:34307 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:34307: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
I20260805 23:18:10.167990 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:10.167756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:44311)
0805 23:18:10.167885 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:10.167934 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260805 23:18:10.168637 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:10.168451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:33391)
0805 23:18:10.168548 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:10.168588 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260805 23:18:10.168841 7603 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:18:10.769599 7607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:10.769129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:56101)
0805 23:18:10.769410 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:10.769479 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260805 23:18:10.973865 7608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:10.973490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:58805)
0805 23:18:10.973727 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:10.973779 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":68,"threads_started":1}
I20260805 23:18:11.169992 7609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.169464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:34155)
0805 23:18:11.169807 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:11.169869 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":78,"threads_started":1}
I20260805 23:18:11.428604 7610 raft_consensus.cc:482] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:11.428822 7610 raft_consensus.cc:504] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:11.430015 7610 leader_election.cc:288] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599), 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945)
Aug 05 23:18:11 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.66@KRBTEST.COM
Aug 05 23:18:11 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971889, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.67@KRBTEST.COM
I20260805 23:18:11.443343 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.430188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35599 (local address 127.23.88.65:36257)
0805 23:18:11.430552 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:11.430569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:11.430680 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:11.431954 (+ 1274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:11.431960 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:11.432570 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:11.433003 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:11.433018 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.434276 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.434280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:11.435435 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:11.435451 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.436183 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.436187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:11.436400 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:11.436949 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:11.436966 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:11.438557 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:11.440924 (+ 2367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:11.440935 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:11.440948 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:11.441208 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:11.441568 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441573 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:11.441575 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:11.441715 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:11.442361 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:11.442367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:11.442601 (+ 234us) client_negotiation.cc:769] Sending connection context
0805 23:18:11.442789 (+ 188us) client_negotiation.cc:240] Negotiation successful
0805 23:18:11.443144 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":134,"spinlock_wait_cycles":3456,"thread_start_us":81,"threads_started":1}
I20260805 23:18:11.443372 7614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.430488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:36257 (local address 127.23.88.66:35599)
0805 23:18:11.431002 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:11.431013 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:11.431043 (+ 30us) server_negotiation.cc:407] Connection header received
0805 23:18:11.431278 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:11.431282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:11.431604 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:11.431859 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:11.433184 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.434130 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.435586 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.436036 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.436218 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:11.438772 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:11.438778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:11.438790 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:11.438817 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:11.440795 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441333 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:11.441339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:11.441345 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:11.441468 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441813 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:11.441817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:11.441819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:11.442241 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:11.442390 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:11.442900 (+ 510us) server_negotiation.cc:299] Negotiation successful
0805 23:18:11.443196 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":90,"threads_started":1}
I20260805 23:18:11.443683 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.430345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:41945 (local address 127.23.88.65:58091)
0805 23:18:11.430653 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:11.430667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:11.430775 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:11.432276 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:11.432278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:11.432557 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:11.433002 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:11.433018 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.434287 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.434290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:11.435433 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:11.435443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.436160 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.436164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:11.436379 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:11.437107 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:11.437119 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:11.438961 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:11.441280 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441284 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:11.441286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:11.441517 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:11.441912 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441917 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:11.441919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:11.442015 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:11.442641 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:11.442645 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:11.442757 (+ 112us) client_negotiation.cc:769] Sending connection context
0805 23:18:11.442882 (+ 125us) client_negotiation.cc:240] Negotiation successful
0805 23:18:11.443233 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":244,"thread_start_us":77,"threads_started":1}
I20260805 23:18:11.444222 7613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.430483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:58091 (local address 127.23.88.67:41945)
0805 23:18:11.430902 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:11.430922 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:11.430964 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:18:11.431672 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:11.431677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:11.432032 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:11.432313 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:11.433185 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.434130 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.435549 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:11.436028 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:11.436192 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:11.439211 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:11.439215 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:11.439226 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:11.439261 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:11.441119 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:11.441650 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:11.441661 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:11.441666 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:11.441801 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:11.442118 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:11.442125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:11.442130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:11.442519 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:11.442671 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:11.443027 (+ 356us) server_negotiation.cc:299] Negotiation successful
0805 23:18:11.443381 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":92,"threads_started":1}
I20260805 23:18:11.444757 7408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "753f97edf1784b20bf998d66d3069107" is_pre_election: true
I20260805 23:18:11.445120 7408 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 1.
I20260805 23:18:11.446290 7193 leader_election.cc:302] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753f97edf1784b20bf998d66d3069107, ab15f3c05d2c4069ab8f2d8ba0e2843c; no voters:
I20260805 23:18:11.446650 7610 raft_consensus.cc:2781] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 23:18:11.446754 7610 raft_consensus.cc:482] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:11.446811 7610 raft_consensus.cc:3037] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 1 FOLLOWER]: Advancing to term 2
W20260805 23:18:11.446904 7192 leader_election.cc:341] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945): Not found: Tablet not found: 9a0c2d3a69b5459db30b823892ca49bb
I20260805 23:18:11.449033 7610 raft_consensus.cc:504] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:11.449422 7610 leader_election.cc:288] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 election: Requested vote from peers 753f97edf1784b20bf998d66d3069107 (127.23.88.66:35599), 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945)
W20260805 23:18:11.450204 7192 leader_election.cc:341] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945): Not found: Tablet not found: 9a0c2d3a69b5459db30b823892ca49bb
I20260805 23:18:11.450177 7408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a0c2d3a69b5459db30b823892ca49bb" candidate_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "753f97edf1784b20bf998d66d3069107"
I20260805 23:18:11.450433 7408 raft_consensus.cc:3037] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:18:11.453158 7408 raft_consensus.cc:2454] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab15f3c05d2c4069ab8f2d8ba0e2843c in term 2.
I20260805 23:18:11.453599 7193 leader_election.cc:302] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 753f97edf1784b20bf998d66d3069107, ab15f3c05d2c4069ab8f2d8ba0e2843c; no voters: 2f74b476538f44cb8a0ee415afbb05b3
I20260805 23:18:11.453821 7610 raft_consensus.cc:2781] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:18:11.454039 7610 raft_consensus.cc:686] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [term 2 LEADER]: Becoming Leader. State: Replica: ab15f3c05d2c4069ab8f2d8ba0e2843c, State: Running, Role: LEADER
I20260805 23:18:11.454412 7610 consensus_queue.cc:237] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:11.455022 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.454750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:49763)
0805 23:18:11.454884 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:11.454953 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:18:11.915624 7192 consensus_peers.cc:487] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c -> Peer 2f74b476538f44cb8a0ee415afbb05b3 (127.23.88.67:41945): Couldn't send request to peer 2f74b476538f44cb8a0ee415afbb05b3. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9a0c2d3a69b5459db30b823892ca49bb. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:18:11.975122 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:11.974589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:60791)
0805 23:18:11.974857 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:11.974974 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":94,"threads_started":1}
I20260805 23:18:12.004710 7408 raft_consensus.cc:1261] T 9a0c2d3a69b5459db30b823892ca49bb P 753f97edf1784b20bf998d66d3069107 [term 2 FOLLOWER]: Refusing update from remote peer ab15f3c05d2c4069ab8f2d8ba0e2843c: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260805 23:18:12.005563 7610 consensus_queue.cc:1035] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c [LEADER]: Connected to new peer: Peer: permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260805 23:18:12.007750 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.007501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:52923)
0805 23:18:12.007622 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.007681 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260805 23:18:12.010483 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.010142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:50243)
0805 23:18:12.010355 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.010405 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
I20260805 23:18:12.010637 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.010379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:47467)
0805 23:18:12.010489 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.010563 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260805 23:18:12.171268 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.170777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:52389)
0805 23:18:12.171050 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.171156 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260805 23:18:12.324766 7626 ts_tablet_manager.cc:927] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Initiating tablet copy from peer ab15f3c05d2c4069ab8f2d8ba0e2843c (127.23.88.65:37607)
I20260805 23:18:12.325378 7626 tablet_copy_client.cc:316] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: tablet copy: Beginning tablet copy session from remote peer at address 127.23.88.65:37607
Aug 05 23:18:12 dist-test-slave-vkjp krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971890, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:18:12.339690 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.325870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:37607 (local address 127.23.88.67:51785)
0805 23:18:12.326024 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.326061 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:12.326195 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:12.326755 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:12.326761 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:12.327227 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:12.327343 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:12.327350 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:12.328362 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:12.328367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:12.328996 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:12.329005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:12.329599 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:12.329603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:12.329721 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:12.334674 (+ 4953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:12.334687 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:12.336123 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:12.337895 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:12.337905 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:12.337911 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:12.338146 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:12.338454 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:12.338459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:12.338464 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:12.338550 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:12.339015 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:12.339022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:12.339193 (+ 171us) client_negotiation.cc:769] Sending connection context
0805 23:18:12.339343 (+ 150us) client_negotiation.cc:240] Negotiation successful
0805 23:18:12.339517 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260805 23:18:12.339731 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.326044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:51785 (local address 127.23.88.65:37607)
0805 23:18:12.326363 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:12.326373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:12.326393 (+ 20us) server_negotiation.cc:407] Connection header received
0805 23:18:12.326437 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:12.326448 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:12.326611 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:12.326754 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:12.327489 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:12.328246 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:12.329121 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:12.329477 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:12.329596 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:12.336276 (+ 6680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:12.336284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:12.336296 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:12.336321 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:12.337781 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:12.338275 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:12.338280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:12.338281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:12.338360 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:12.338644 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:12.338648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:12.338650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:12.338896 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:12.339042 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:12.339427 (+ 385us) server_negotiation.cc:299] Negotiation successful
0805 23:18:12.339580 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260805 23:18:12.340827 7283 tablet_copy_service.cc:140] P ab15f3c05d2c4069ab8f2d8ba0e2843c: Received BeginTabletCopySession request for tablet 9a0c2d3a69b5459db30b823892ca49bb from peer 2f74b476538f44cb8a0ee415afbb05b3 ({username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:51785)
I20260805 23:18:12.341045 7283 tablet_copy_service.cc:161] P ab15f3c05d2c4069ab8f2d8ba0e2843c: Beginning new tablet copy session on tablet 9a0c2d3a69b5459db30b823892ca49bb from peer 2f74b476538f44cb8a0ee415afbb05b3 at {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:51785: session id = 2f74b476538f44cb8a0ee415afbb05b3-9a0c2d3a69b5459db30b823892ca49bb
I20260805 23:18:12.342178 7283 tablet_copy_source_session.cc:205] T 9a0c2d3a69b5459db30b823892ca49bb P ab15f3c05d2c4069ab8f2d8ba0e2843c: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:18:12.346081 7626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a0c2d3a69b5459db30b823892ca49bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:12.352450 7626 tablet_copy_client.cc:791] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: tablet copy: Starting download of 0 data blocks...
I20260805 23:18:12.352685 7626 tablet_copy_client.cc:655] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: tablet copy: Starting download of 1 WAL segments...
I20260805 23:18:12.354506 7626 tablet_copy_client.cc:531] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:18:12.356417 7626 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Bootstrap starting.
I20260805 23:18:12.361670 7626 log.cc:824] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:12.368043 7626 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:12.368336 7626 tablet_bootstrap.cc:492] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Bootstrap complete.
I20260805 23:18:12.368683 7626 ts_tablet_manager.cc:1397] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Time spent bootstrapping tablet: real 0.012s user 0.012s sys 0.000s
I20260805 23:18:12.370893 7626 raft_consensus.cc:348] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:12.371184 7626 raft_consensus.cc:729] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f74b476538f44cb8a0ee415afbb05b3, State: Initialized, Role: FOLLOWER
I20260805 23:18:12.371574 7626 consensus_queue.cc:260] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753f97edf1784b20bf998d66d3069107" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35599 } } peers { permanent_uuid: "2f74b476538f44cb8a0ee415afbb05b3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 41945 } } peers { permanent_uuid: "ab15f3c05d2c4069ab8f2d8ba0e2843c" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 37607 } }
I20260805 23:18:12.372464 7626 ts_tablet_manager.cc:1428] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260805 23:18:12.372745 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.372463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:40349)
0805 23:18:12.372605 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.372664 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 23:18:12.373190 7283 tablet_copy_service.cc:342] P ab15f3c05d2c4069ab8f2d8ba0e2843c: Request end of tablet copy session 2f74b476538f44cb8a0ee415afbb05b3-9a0c2d3a69b5459db30b823892ca49bb received from {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:51785
I20260805 23:18:12.373318 7283 tablet_copy_service.cc:433] P ab15f3c05d2c4069ab8f2d8ba0e2843c: ending tablet copy session 2f74b476538f44cb8a0ee415afbb05b3-9a0c2d3a69b5459db30b823892ca49bb on tablet 9a0c2d3a69b5459db30b823892ca49bb with peer 2f74b476538f44cb8a0ee415afbb05b3
I20260805 23:18:12.373466 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.373191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:54593)
0805 23:18:12.373322 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.373371 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260805 23:18:12.774471 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.774115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:54621)
0805 23:18:12.774325 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.774372 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":50,"threads_started":1}
I20260805 23:18:12.774495 7552 raft_consensus.cc:1203] T 9a0c2d3a69b5459db30b823892ca49bb P 2f74b476538f44cb8a0ee415afbb05b3 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260805 23:18:12.775620 7625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:12.775308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:44835)
0805 23:18:12.775466 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:12.775529 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260805 23:18:13.011909 7636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:13.011420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:55633)
0805 23:18:13.011722 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:13.011791 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":74,"threads_started":1}
I20260805 23:18:13.775880 7637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:13.775452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:37245)
0805 23:18:13.775712 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:13.775781 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260805 23:18:13.776718 7638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:13.776316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:46347)
0805 23:18:13.776576 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:13.776631 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":63,"threads_started":1}
I20260805 23:18:14.013168 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:14.012716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:58721)
0805 23:18:14.013009 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:14.013069 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":81,"threads_started":1}
I20260805 23:18:14.777433 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:14.776967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:51495)
0805 23:18:14.777270 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:14.777335 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":102,"threads_started":1}
I20260805 23:18:14.777835 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:14.777399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:59233)
0805 23:18:14.777705 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:14.777756 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
I20260805 23:18:14.778280 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:14.778059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:34567)
0805 23:18:14.778173 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:14.778224 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260805 23:18:15.014822 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:15.014316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:38855)
0805 23:18:15.014636 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:15.014704 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":111,"threads_started":1}
I20260805 23:18:15.779351 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:15.778883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:46045)
0805 23:18:15.779196 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:15.779255 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":61,"threads_started":1}
I20260805 23:18:15.779407 7643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:15.778819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:49365)
0805 23:18:15.779219 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:15.779295 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":114,"threads_started":1}
I20260805 23:18:16.016112 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:16.015651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:35489)
0805 23:18:16.015953 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:16.016012 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":91,"threads_started":1}
I20260805 23:18:16.780663 7647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:16.780221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:33705)
0805 23:18:16.780501 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:16.780564 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260805 23:18:16.780660 7646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:16.780171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:54433)
0805 23:18:16.780512 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:16.780570 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":135,"threads_started":1}
I20260805 23:18:17.017401 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:17.016944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:37385)
0805 23:18:17.017242 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:17.017304 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
I20260805 23:18:17.781908 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:17.781492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:42983)
0805 23:18:17.781753 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:17.781812 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":97,"threads_started":1}
I20260805 23:18:17.781972 7652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:17.781568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:58203)
0805 23:18:17.781824 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:17.781881 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":93,"threads_started":1}
I20260805 23:18:18.019047 7653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:18.018344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:37991)
0805 23:18:18.018796 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:18.018890 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":219,"threads_started":1}
I20260805 23:18:18.783262 7654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:18.782777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:41227)
0805 23:18:18.783052 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:18.783163 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":107,"threads_started":1}
I20260805 23:18:18.783265 7655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:18.782770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:54643)
0805 23:18:18.783063 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:18.783172 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":120,"threads_started":1}
I20260805 23:18:19.020308 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:19.019873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:55295)
0805 23:18:19.020151 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:19.020209 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":94,"threads_started":1}
I20260805 23:18:19.784436 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:19.784022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:47711)
0805 23:18:19.784282 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:19.784336 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":97,"threads_started":1}
I20260805 23:18:19.784644 7658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:19.784190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:54379)
0805 23:18:19.784456 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:19.784533 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":108,"threads_started":1}
I20260805 23:18:20.021716 7661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:20.021258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:55189)
0805 23:18:20.021568 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:20.021622 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260805 23:18:20.785806 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:20.785372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:53207)
0805 23:18:20.785646 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:20.785707 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":86,"threads_started":1}
I20260805 23:18:20.785883 7664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:20.785372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:37179)
0805 23:18:20.785722 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:20.785776 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":99,"threads_started":1}
I20260805 23:18:21.023198 7665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:21.022638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:34139)
0805 23:18:21.022982 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:21.023037 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":151,"threads_started":1}
I20260805 23:18:21.787228 7667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:21.786709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:33391)
0805 23:18:21.786988 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:21.787061 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
I20260805 23:18:21.787479 7666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:21.786709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:51903)
0805 23:18:21.787272 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:21.787362 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":99,"threads_started":1}
I20260805 23:18:22.024564 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:22.024079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:46245)
0805 23:18:22.024381 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:22.024453 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260805 23:18:22.788590 7669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:22.788038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:56569)
0805 23:18:22.788364 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:22.788446 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260805 23:18:22.788841 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:22.788369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:49999)
0805 23:18:22.788644 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:22.788727 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":98,"threads_started":1}
I20260805 23:18:23.025960 7671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:23.025508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:57041)
0805 23:18:23.025794 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:23.025860 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":84,"threads_started":1}
I20260805 23:18:23.789922 7672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:23.789483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:56249)
0805 23:18:23.789764 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:23.789826 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":104,"threads_started":1}
I20260805 23:18:23.790079 7673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:23.789641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:34925)
0805 23:18:23.789899 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:23.789961 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":92,"threads_started":1}
I20260805 23:18:24.027688 7674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:24.027178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:44867)
0805 23:18:24.027527 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:24.027587 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":76,"threads_started":1}
I20260805 23:18:24.791488 7675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:24.791006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.67:54101)
0805 23:18:24.791334 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:24.791392 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":120,"threads_started":1}
I20260805 23:18:24.791517 7676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:24.791017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.65:53405)
0805 23:18:24.791350 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:24.791424 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":150,"threads_started":1}
I20260805 23:18:25.029309 7678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:25.028848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34307 (local address 127.23.88.66:37427)
0805 23:18:25.029149 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:25.029210 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:34307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":114,"threads_started":1}
I20260805 23:18:25.173611 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7134
I20260805 23:18:25.180253 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7316
I20260805 23:18:25.187325 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7464
2026-08-05T23:18:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19604 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:18:25 dist-test-slave-vkjp krb5kdc[7681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:18:25 dist-test-slave-vkjp krb5kdc[7681](info): set up 2 sockets
Aug 05 23:18:25 dist-test-slave-vkjp krb5kdc[7681](info): commencing operation
krb5kdc: starting...
W20260805 23:18:26.260665 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:18:26 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:18:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:18:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:18:26.395679 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:26.508603 7694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:26.508970 7694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:26.509050 7694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:26.513360 7694 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:18:26.513460 7694 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:18:26.513486 7694 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:18:26.513509 7694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:26.513554 7694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:26.513599 7694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:26.513654 7694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:26.513696 7694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:26.519405 7694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:45211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:26.520648 7694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:26.521818 7694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:18:26.530972 7694 server_base.cc:1034] running on GCE node
W20260805 23:18:26.530920 7703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:26.530942 7700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:26.530920 7699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:26.531845 7694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:26.533057 7694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:26.534268 7694 hybrid_clock.cc:648] HybridClock initialized: now 1785971906534257 us; error 41 us; skew 500 ppm
Aug 05 23:18:26 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971906, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:26.541558 7694 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:26.542640 7694 webserver.cc:460] Webserver started at http://127.23.88.126:42855/ using document root <none> and password file <none>
I20260805 23:18:26.543329 7694 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:26.543433 7694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:26.543685 7694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:26.545841 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "ccafc7c8f879436c9c28401b920d285b"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "49e18cc45fd5b9808181317a6a493bc0"
server_key_iv: "7dbee7a8537a98ed91dbee165878cf37"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.546461 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "ccafc7c8f879436c9c28401b920d285b"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "49e18cc45fd5b9808181317a6a493bc0"
server_key_iv: "7dbee7a8537a98ed91dbee165878cf37"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.550534 7694 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:18:26.553222 7710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:26.554594 7694 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260805 23:18:26.554749 7694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "ccafc7c8f879436c9c28401b920d285b"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "49e18cc45fd5b9808181317a6a493bc0"
server_key_iv: "7dbee7a8537a98ed91dbee165878cf37"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.554877 7694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:26.581737 7694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:26.585165 7694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:26.585408 7694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:26.593004 7694 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:45211
I20260805 23:18:26.593381 7764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:45211 every 8 connection(s)
I20260805 23:18:26.594074 7694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:26.596909 7767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:26.602918 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap starting.
I20260805 23:18:26.603400 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7694
I20260805 23:18:26.603554 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:18:26.603842 23905 external_mini_cluster.cc:1428] Setting key 63cba6ee75ff93aaabab1b50406311ea
I20260805 23:18:26.605907 7767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:26.606719 7767 log.cc:824] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:26.608882 7767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: No bootstrap required, opened a new log
I20260805 23:18:26.612766 7767 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:26.613008 7767 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:26.613044 7767 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Initialized, Role: FOLLOWER
I20260805 23:18:26.613631 7767 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:26.613827 7767 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:26.613890 7767 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:26.613994 7767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 0 FOLLOWER]: Advancing to term 1
Aug 05 23:18:26 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:26.614982 7767 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:26.615377 7767 leader_election.cc:302] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccafc7c8f879436c9c28401b920d285b; no voters:
I20260805 23:18:26.615736 7767 leader_election.cc:288] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:18:26.616174 7773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:26.616492 7773 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 LEADER]: Becoming Leader. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Running, Role: LEADER
I20260805 23:18:26.616918 7773 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:26.617491 7767 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:26.618809 7774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:18:26.618947 7774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:26.619416 7775 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccafc7c8f879436c9c28401b920d285b. Latest consensus state: current_term: 1 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:18:26.619604 7775 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:26.621628 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:26.605335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44138 (local address 127.23.88.126:45211)
0805 23:18:26.605851 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:26.605875 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:26.605920 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:18:26.606763 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:26.606784 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:26.607353 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:26.607742 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:26.608676 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.609795 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.610775 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.611410 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.611613 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:26.614767 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:26.614774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:26.614789 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:26.614824 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:26.617383 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:26.617984 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:26.617991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:26.617999 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:26.618167 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:26.618644 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:26.618650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:26.618653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:26.619315 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:26.620280 (+ 965us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:26.620483 (+ 203us) server_negotiation.cc:299] Negotiation successful
0805 23:18:26.620781 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":140,"threads_started":1}
W20260805 23:18:26.623139 7789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:18:26.623234 7789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:18:26.623323 7783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:26.624059 7783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:26.629626 7783 catalog_manager.cc:1347] Generated new cluster ID: 1f2f9eab49b8416ebdb3f8e8327eb454
I20260805 23:18:26.629716 7783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:26.646531 7783 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:18:26.647925 7783 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:26.659503 7783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Generated new TSK 0
I20260805 23:18:26.660465 7783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:18:26.699040 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:26.820122 7796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:26.820427 7796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:26.820499 7796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:26.824736 7796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:26.824818 7796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:26.824882 7796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:18:26.830930 7796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:26.832199 7796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:26.833362 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:26.843703 7802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:26.843703 7803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:26.843709 7805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:26.844017 7796 server_base.cc:1034] running on GCE node
I20260805 23:18:26.844575 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:26.845250 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:26.846459 7796 hybrid_clock.cc:648] HybridClock initialized: now 1785971906846443 us; error 27 us; skew 500 ppm
Aug 05 23:18:26 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971906, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:26.849465 7796 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:18:26.850513 7796 webserver.cc:460] Webserver started at http://127.23.88.65:35507/ using document root <none> and password file <none>
I20260805 23:18:26.851380 7796 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:26.851501 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:26.851804 7796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:26.854405 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "210606a0859c4995ac2213026d78cd67"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "374f01627c9af3fa85eebcaac0b9dfa7"
server_key_iv: "d02de52a852527a4da04bbfce58cdeba"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.855170 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "210606a0859c4995ac2213026d78cd67"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "374f01627c9af3fa85eebcaac0b9dfa7"
server_key_iv: "d02de52a852527a4da04bbfce58cdeba"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.860939 7796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260805 23:18:26.863847 7813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:26.865499 7796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:18:26.865667 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "210606a0859c4995ac2213026d78cd67"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "374f01627c9af3fa85eebcaac0b9dfa7"
server_key_iv: "d02de52a852527a4da04bbfce58cdeba"
server_key_version: "encryptionkey@0"
I20260805 23:18:26.865808 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:26.879822 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:26.883219 7796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:26.883437 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:26.884078 7796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:26.884960 7796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:26.885035 7796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:26.885108 7796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:26.885160 7796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:26.894074 7796 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:41377
I20260805 23:18:26.894124 7940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:41377 every 8 connection(s)
I20260805 23:18:26.895071 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:18:26.897429 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7796
I20260805 23:18:26.897590 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:18:26.897935 23905 external_mini_cluster.cc:1428] Setting key 1d652b4856b0d9d0afc49680ea93f58d
Aug 05 23:18:26 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971906, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:18:26.911175 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:26.896997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:51163 (local address 127.23.88.126:45211)
0805 23:18:26.897215 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:26.897221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:26.898186 (+ 965us) server_negotiation.cc:407] Connection header received
0805 23:18:26.899886 (+ 1700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:26.899892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:26.899951 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:26.900052 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:26.901496 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.902069 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.903410 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.903648 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.903751 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:26.906825 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:26.906832 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:26.906838 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:26.906889 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:26.908768 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:26.909411 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:26.909415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:26.909417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:26.909521 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:26.909946 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:26.909949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:26.909950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:26.910151 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:26.910235 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:26.910830 (+ 595us) server_negotiation.cc:299] Negotiation successful
0805 23:18:26.910973 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260805 23:18:26.912153 7943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:26.897384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:51163)
0805 23:18:26.898001 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:26.898060 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:26.899652 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:26.900220 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:26.900227 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:26.900624 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:26.901280 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:26.901304 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.902218 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.902225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:26.903271 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:26.903285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:26.903829 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:26.903835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:26.904080 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:26.904785 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:26.904813 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:26.906613 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:26.908957 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:26.908963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:26.908983 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:26.909297 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:26.909659 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:26.909664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:26.909669 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:26.909842 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:26.910251 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:26.910256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:26.910545 (+ 289us) client_negotiation.cc:769] Sending connection context
0805 23:18:26.910778 (+ 233us) client_negotiation.cc:240] Negotiation successful
0805 23:18:26.911071 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":183,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:18:26.913637 7941 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:26.913914 7941 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:26.914527 7941 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:26.916216 7729 ts_manager.cc:194] Registered new tserver with Master: 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:26.917459 7729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:51163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:18:26.930696 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:27.043881 7947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:27.044173 7947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:27.044219 7947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:27.048403 7947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:27.048465 7947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:27.048568 7947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:18:27.054268 7947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:27.055469 7947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:27.056525 7947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:27.066969 7956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:27.066951 7953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:27.066946 7954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:27.067152 7947 server_base.cc:1034] running on GCE node
I20260805 23:18:27.067731 7947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:27.068380 7947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:27.069579 7947 hybrid_clock.cc:648] HybridClock initialized: now 1785971907069559 us; error 30 us; skew 500 ppm
Aug 05 23:18:27 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971907, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:27.072738 7947 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:18:27.073720 7947 webserver.cc:460] Webserver started at http://127.23.88.66:44663/ using document root <none> and password file <none>
I20260805 23:18:27.074335 7947 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:27.074429 7947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:27.074657 7947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:27.076833 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "ef24a29e52744650bbe374199cd06f0f"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "6f753962ead8361e82973020366d5297"
server_key_iv: "9d04a53fbd4e8f3181a8634368b61dd9"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.077457 7947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "ef24a29e52744650bbe374199cd06f0f"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "6f753962ead8361e82973020366d5297"
server_key_iv: "9d04a53fbd4e8f3181a8634368b61dd9"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.081364 7947 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:18:27.083950 7965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.085256 7947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:18:27.085390 7947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "ef24a29e52744650bbe374199cd06f0f"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "6f753962ead8361e82973020366d5297"
server_key_iv: "9d04a53fbd4e8f3181a8634368b61dd9"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.085510 7947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:27.104107 7947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:27.107578 7947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:27.107806 7947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:27.108426 7947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:27.109277 7947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:27.109349 7947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.109416 7947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:27.109458 7947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.118958 7947 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35403
I20260805 23:18:27.118980 8088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35403 every 8 connection(s)
I20260805 23:18:27.120030 7947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:18:27.128131 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 7947
I20260805 23:18:27.128289 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:18:27.128677 23905 external_mini_cluster.cc:1428] Setting key 455f1348c0f21c34a8bd1a0a1c4778bd
Aug 05 23:18:27 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971907, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:27.134912 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.121976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:35249 (local address 127.23.88.126:45211)
0805 23:18:27.122118 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.122122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.122970 (+ 848us) server_negotiation.cc:407] Connection header received
0805 23:18:27.124444 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.124448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.124511 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.124613 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:27.125807 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.126351 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.127556 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.127784 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.127907 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.130676 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:27.130682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:27.130688 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:27.130720 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:27.132384 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.133070 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.133073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.133075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.133159 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.133587 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.133593 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.133597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.133888 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:27.133999 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.134506 (+ 507us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.134668 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:18:27.135766 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.122354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:35249)
0805 23:18:27.122811 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.122863 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.124182 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.124787 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.124794 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.125178 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:27.125662 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.125675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.126489 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.126493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.127435 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.127444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.127960 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.127964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.128125 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.128854 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:27.128871 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:27.130498 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:27.132505 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:27.132513 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:27.132532 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:27.132940 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:27.133261 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:27.133265 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:27.133269 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:27.133462 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:27.133996 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:27.134001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:27.134250 (+ 249us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.134443 (+ 193us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.134713 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260805 23:18:27.137076 8089 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:27.137473 8089 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:27.138144 8089 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:27.139496 7729 ts_manager.cc:194] Registered new tserver with Master: ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403)
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:18:27.140156 7729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:35249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:18:27.159302 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:27.273523 8094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:27.273900 8094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:27.274032 8094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:27.278326 8094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:27.278437 8094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:27.278537 8094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:18:27.284168 8094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:27.285436 8094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:27.286499 8094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:27.296733 8102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:27.296942 8094 server_base.cc:1034] running on GCE node
W20260805 23:18:27.296715 8104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:27.296715 8101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:27.297581 8094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:27.298291 8094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:27.299490 8094 hybrid_clock.cc:648] HybridClock initialized: now 1785971907299476 us; error 30 us; skew 500 ppm
Aug 05 23:18:27 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971907, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:27.302609 8094 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:18:27.303714 8094 webserver.cc:460] Webserver started at http://127.23.88.67:43863/ using document root <none> and password file <none>
I20260805 23:18:27.304333 8094 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:27.304401 8094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:27.304636 8094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:18:27.306872 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "d1d2521331e74439bc69f26fec2b6194"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "a1f3d4e4c806b45327771da78fd13a31"
server_key_iv: "fe002d9a9db5eb9907b66c593b96437a"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.307564 8094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "d1d2521331e74439bc69f26fec2b6194"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "a1f3d4e4c806b45327771da78fd13a31"
server_key_iv: "fe002d9a9db5eb9907b66c593b96437a"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.312341 8094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:18:27.315047 8112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.316669 8094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:27.316819 8094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "d1d2521331e74439bc69f26fec2b6194"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "a1f3d4e4c806b45327771da78fd13a31"
server_key_iv: "fe002d9a9db5eb9907b66c593b96437a"
server_key_version: "encryptionkey@0"
I20260805 23:18:27.316952 8094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:27.342051 8094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:27.345728 8094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:27.345952 8094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:27.346627 8094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:27.347613 8094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:27.347663 8094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.347728 8094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:27.347760 8094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:27.356686 8094 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:39037
I20260805 23:18:27.356775 8234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:39037 every 8 connection(s)
I20260805 23:18:27.357688 8094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:18:27.366721 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8094
I20260805 23:18:27.366894 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:18:27.367259 23905 external_mini_cluster.cc:1428] Setting key 8bd9fecee22c9e790d5d378da5fb101b
Aug 05 23:18:27 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971907, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:27.372943 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.359703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:43693 (local address 127.23.88.126:45211)
0805 23:18:27.359874 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.359877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.360802 (+ 925us) server_negotiation.cc:407] Connection header received
0805 23:18:27.362204 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.362207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.362271 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.362365 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:27.363681 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.364235 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.365390 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.365630 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.365754 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.368802 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:27.368806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:27.368809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:27.368842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:27.370555 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.371148 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.371152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.371154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.371253 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.371638 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.371643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.371645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.371858 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:27.371972 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.372581 (+ 609us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.372757 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:18:27.373816 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.360175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:43693)
0805 23:18:27.360649 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.360693 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.361974 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.362535 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.362541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.362909 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:27.363532 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.363546 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.364369 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.364372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.365257 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.365266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.365842 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.365848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.366020 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.366660 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:27.366684 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:27.368614 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:27.370671 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:27.370678 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:27.370696 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:27.370991 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:27.371357 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:27.371361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:27.371363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:27.371535 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:27.371973 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:27.371982 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:27.372264 (+ 282us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.372495 (+ 231us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.372889 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":125,"threads_started":1}
I20260805 23:18:27.375264 8235 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:27.375554 8235 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:27.376113 8235 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:27.377290 7729 ts_manager.cc:194] Registered new tserver with Master: d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037)
I20260805 23:18:27.377908 7729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:43693
I20260805 23:18:27.381682 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:18:27.391403 7771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.383818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.23.88.126:45211)
0805 23:18:27.383957 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.383961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.384135 (+ 174us) server_negotiation.cc:407] Connection header received
0805 23:18:27.384242 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.384244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.384288 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.384364 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:27.385016 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.385549 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.386358 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.386586 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.386665 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.388085 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:27.388088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:27.388090 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:27.388115 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:27.389691 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.390136 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.390139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.390141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.390219 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:27.390486 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:27.390489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:27.390490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:27.390742 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:27.390861 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.391073 (+ 212us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.391263 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:18:27.394531 7729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:18:27.397919 7729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:18:27.414003 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.408423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.23.88.67:39037)
0805 23:18:27.408811 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.408821 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.408840 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:18:27.408906 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.408910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.409082 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.409312 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:27.409526 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410737 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.412475 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.413230 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.413330 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.413463 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.413718 (+ 255us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.413833 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":138,"threads_started":1}
I20260805 23:18:27.414065 8251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.407944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39037 (local address 127.0.0.1:39822)
0805 23:18:27.408456 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.408469 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.408557 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.409223 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.409225 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.409241 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:27.409421 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.409428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410910 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.410914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.412333 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.412341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.413359 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.413363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.413500 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.413521 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.413666 (+ 145us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.413895 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":340,"thread_start_us":114,"threads_started":1}
I20260805 23:18:27.414527 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.407384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.23.88.65:41377)
0805 23:18:27.408131 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.408148 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.408596 (+ 448us) server_negotiation.cc:407] Connection header received
0805 23:18:27.408689 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.408695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.408893 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.409068 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:27.409434 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410254 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.412203 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.413078 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.413202 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.413389 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.414238 (+ 849us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.414346 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":64,"threads_started":1}
I20260805 23:18:27.414866 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.407079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35403 (local address 127.0.0.1:37132)
0805 23:18:27.407992 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.408054 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.408185 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.409080 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.409083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.409102 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:27.409286 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.409294 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410912 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.410915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.412323 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.412341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.414473 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.414477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.414585 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.414618 (+ 33us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.414673 (+ 55us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.414754 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":91,"thread_start_us":128,"threads_started":1}
I20260805 23:18:27.415190 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.407136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.0.0.1:55570)
0805 23:18:27.408288 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.408302 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.408407 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.409061 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.409067 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.409089 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:27.409260 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.409268 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410447 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.410453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.412063 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.412073 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.413222 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.413226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.413436 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.413520 (+ 84us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.413666 (+ 146us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.413870 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1084,"mutex_wait_us":126,"thread_start_us":87,"threads_started":1}
I20260805 23:18:27.415606 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.407218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.23.88.66:35403)
0805 23:18:27.408561 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.408575 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.408603 (+ 28us) server_negotiation.cc:407] Connection header received
0805 23:18:27.408691 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.408696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.408917 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.409434 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:27.409615 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.410750 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.413477 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.414327 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.415064 (+ 737us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.415264 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.415323 (+ 59us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.415439 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":115,"threads_started":1}
I20260805 23:18:27.418017 7866 tablet_service.cc:1467] Processing CreateTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 (DEFAULT_TABLE table=test-table [id=2606a329b4154df2bc4e0736c45a083f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:27.418174 8164 tablet_service.cc:1467] Processing CreateTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 (DEFAULT_TABLE table=test-table [id=2606a329b4154df2bc4e0736c45a083f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:27.419343 7866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf21e3f3d6f84065baeebbc9a9529200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:27.419394 8164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf21e3f3d6f84065baeebbc9a9529200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:27.418953 8019 tablet_service.cc:1467] Processing CreateTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 (DEFAULT_TABLE table=test-table [id=2606a329b4154df2bc4e0736c45a083f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:27.420099 8019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf21e3f3d6f84065baeebbc9a9529200. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:27.426105 8253 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Bootstrap starting.
I20260805 23:18:27.428284 8255 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Bootstrap starting.
I20260805 23:18:27.428284 8254 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Bootstrap starting.
I20260805 23:18:27.428452 8253 tablet_bootstrap.cc:654] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:27.429183 8253 log.cc:824] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:27.430768 8254 tablet_bootstrap.cc:654] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:27.431404 8255 tablet_bootstrap.cc:654] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:27.432312 8254 log.cc:824] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:27.432324 8255 log.cc:824] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:27.432760 8253 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: No bootstrap required, opened a new log
I20260805 23:18:27.432976 8253 ts_tablet_manager.cc:1397] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260805 23:18:27.434243 8255 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: No bootstrap required, opened a new log
I20260805 23:18:27.434439 8254 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: No bootstrap required, opened a new log
I20260805 23:18:27.434476 8255 ts_tablet_manager.cc:1397] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:18:27.434643 8254 ts_tablet_manager.cc:1397] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:18:27.437666 8253 raft_consensus.cc:348] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.437969 8253 raft_consensus.cc:374] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:27.438031 8253 raft_consensus.cc:729] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210606a0859c4995ac2213026d78cd67, State: Initialized, Role: FOLLOWER
I20260805 23:18:27.438030 8255 raft_consensus.cc:348] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.438364 8255 raft_consensus.cc:374] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:27.438434 8255 raft_consensus.cc:729] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef24a29e52744650bbe374199cd06f0f, State: Initialized, Role: FOLLOWER
I20260805 23:18:27.438508 8253 consensus_queue.cc:260] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.438421 8254 raft_consensus.cc:348] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.438659 8254 raft_consensus.cc:374] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:27.438723 8254 raft_consensus.cc:729] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d2521331e74439bc69f26fec2b6194, State: Initialized, Role: FOLLOWER
I20260805 23:18:27.438966 8255 consensus_queue.cc:260] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.439196 8254 consensus_queue.cc:260] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.439388 7941 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
I20260805 23:18:27.439651 8253 ts_tablet_manager.cc:1428] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260805 23:18:27.440021 8255 ts_tablet_manager.cc:1428] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20260805 23:18:27.440140 8254 ts_tablet_manager.cc:1428] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260805 23:18:27.440210 8235 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
I20260805 23:18:27.440429 8089 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
I20260805 23:18:27.441350 8261 raft_consensus.cc:482] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:27.441526 8261 raft_consensus.cc:504] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.442692 8261 leader_election.cc:288] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403), 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:27.447564 8249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.443241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:50987 (local address 127.23.88.66:35403)
0805 23:18:27.443504 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.443509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.443523 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:18:27.443569 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.443571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.443632 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.443704 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:27.444111 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.444838 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.446210 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.446815 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.446957 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.447019 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.447369 (+ 350us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.447448 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260805 23:18:27.447564 8237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.442934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35403 (local address 127.23.88.67:50987)
0805 23:18:27.443221 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.443238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.443331 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.443824 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.443827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.443848 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:27.443980 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.443987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.444988 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.444992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.446043 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.446052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.447106 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.447110 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.447218 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.447242 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.447351 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.447427 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260805 23:18:27.448206 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.442949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.23.88.67:53957)
0805 23:18:27.443632 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.443649 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:27.443815 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:27.444209 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:27.444212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:27.444229 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:27.444375 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.444382 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.445489 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.445492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.446679 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:27.446694 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.447764 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.447770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:27.447884 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.447905 (+ 21us) client_negotiation.cc:769] Sending connection context
0805 23:18:27.448020 (+ 115us) client_negotiation.cc:240] Negotiation successful
0805 23:18:27.448086 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":354,"thread_start_us":122,"threads_started":1}
I20260805 23:18:27.448267 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.443056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:53957 (local address 127.23.88.65:41377)
0805 23:18:27.443607 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.443611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.443729 (+ 118us) server_negotiation.cc:407] Connection header received
0805 23:18:27.443953 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.443956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.444006 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.444060 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:27.444528 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.445341 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.446843 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.447476 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.447559 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.447597 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.448061 (+ 464us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.448140 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468}
I20260805 23:18:27.448202 8041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf21e3f3d6f84065baeebbc9a9529200" candidate_uuid: "d1d2521331e74439bc69f26fec2b6194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef24a29e52744650bbe374199cd06f0f" is_pre_election: true
I20260805 23:18:27.448510 8041 raft_consensus.cc:2454] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d2521331e74439bc69f26fec2b6194 in term 0.
I20260805 23:18:27.449023 8115 leader_election.cc:302] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1d2521331e74439bc69f26fec2b6194, ef24a29e52744650bbe374199cd06f0f; no voters:
I20260805 23:18:27.449213 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf21e3f3d6f84065baeebbc9a9529200" candidate_uuid: "d1d2521331e74439bc69f26fec2b6194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210606a0859c4995ac2213026d78cd67" is_pre_election: true
I20260805 23:18:27.449327 8261 raft_consensus.cc:2781] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:18:27.449450 8261 raft_consensus.cc:482] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:27.449506 8261 raft_consensus.cc:3037] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:27.449518 7888 raft_consensus.cc:2454] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1d2521331e74439bc69f26fec2b6194 in term 0.
I20260805 23:18:27.450641 8261 raft_consensus.cc:504] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.451002 8261 leader_election.cc:288] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [CANDIDATE]: Term 1 election: Requested vote from peers ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403), 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:27.451431 8041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf21e3f3d6f84065baeebbc9a9529200" candidate_uuid: "d1d2521331e74439bc69f26fec2b6194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef24a29e52744650bbe374199cd06f0f"
I20260805 23:18:27.451586 8041 raft_consensus.cc:3037] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:27.451642 7888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf21e3f3d6f84065baeebbc9a9529200" candidate_uuid: "d1d2521331e74439bc69f26fec2b6194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210606a0859c4995ac2213026d78cd67"
I20260805 23:18:27.451836 7888 raft_consensus.cc:3037] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:27.452674 8041 raft_consensus.cc:2454] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d2521331e74439bc69f26fec2b6194 in term 1.
I20260805 23:18:27.453027 8115 leader_election.cc:302] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1d2521331e74439bc69f26fec2b6194, ef24a29e52744650bbe374199cd06f0f; no voters:
I20260805 23:18:27.453085 7888 raft_consensus.cc:2454] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1d2521331e74439bc69f26fec2b6194 in term 1.
I20260805 23:18:27.453249 8261 raft_consensus.cc:2781] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:27.453519 8261 raft_consensus.cc:686] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 1 LEADER]: Becoming Leader. State: Replica: d1d2521331e74439bc69f26fec2b6194, State: Running, Role: LEADER
I20260805 23:18:27.453856 8261 consensus_queue.cc:237] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:27.457489 7729 catalog_manager.cc:5495] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1d2521331e74439bc69f26fec2b6194 (127.23.88.67). New cstate: current_term: 1 leader_uuid: "d1d2521331e74439bc69f26fec2b6194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } health_report { overall_health: HEALTHY } } }
I20260805 23:18:27.479043 8252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.475432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.23.88.67:39037)
0805 23:18:27.475580 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:27.475587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:27.475604 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:18:27.475697 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:27.475700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:27.475754 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:27.475867 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:27.476139 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.476678 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.477545 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:27.477770 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:27.477879 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:27.478195 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:27.478651 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:18:27.478789 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:27.478829 (+ 40us) server_negotiation.cc:299] Negotiation successful
0805 23:18:27.478888 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:18:27.485749 7888 raft_consensus.cc:1261] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Refusing update from remote peer d1d2521331e74439bc69f26fec2b6194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:27.485749 8041 raft_consensus.cc:1261] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Refusing update from remote peer d1d2521331e74439bc69f26fec2b6194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:27.486606 8263 consensus_queue.cc:1035] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:27.486845 8261 consensus_queue.cc:1035] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:27.496193 8268 mvcc.cc:205] Tried to move back new op lower bound from 7315340933053710336 to 7315340932934918144. Current Snapshot: MvccSnapshot[applied={T|T < 7315340933053710336}]
I20260805 23:18:27.498901 8269 mvcc.cc:205] Tried to move back new op lower bound from 7315340933053710336 to 7315340932934918144. Current Snapshot: MvccSnapshot[applied={T|T < 7315340933053710336}]
I20260805 23:18:27.503393 7729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:27.503554 7729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:27.506060 7729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Sending DeleteTablet for 3 replicas of tablet bf21e3f3d6f84065baeebbc9a9529200
I20260805 23:18:27.506893 7866 tablet_service.cc:1514] Processing DeleteTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:27 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55570
I20260805 23:18:27.506902 8164 tablet_service.cc:1514] Processing DeleteTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:27 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39822
I20260805 23:18:27.507470 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7796
I20260805 23:18:27.507490 8019 tablet_service.cc:1514] Processing DeleteTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:27 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:37132
I20260805 23:18:27.509599 8277 tablet_replica.cc:330] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: stopping tablet replica
I20260805 23:18:27.509644 8278 tablet_replica.cc:330] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: stopping tablet replica
I20260805 23:18:27.509939 8278 raft_consensus.cc:2229] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:27.509939 8277 raft_consensus.cc:2229] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:18:27.510255 8278 raft_consensus.cc:2258] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:27.510273 8277 raft_consensus.cc:2258] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:27.511842 8277 ts_tablet_manager.cc:1905] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:27.512400 8278 ts_tablet_manager.cc:1905] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:27.514918 8277 ts_tablet_manager.cc:1918] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:27.515020 8277 log.cc:1196] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/bf21e3f3d6f84065baeebbc9a9529200
I20260805 23:18:27.515038 8278 ts_tablet_manager.cc:1918] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:27.515149 8278 log.cc:1196] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/bf21e3f3d6f84065baeebbc9a9529200
I20260805 23:18:27.515447 8277 ts_tablet_manager.cc:1939] T bf21e3f3d6f84065baeebbc9a9529200 P d1d2521331e74439bc69f26fec2b6194: Deleting consensus metadata
I20260805 23:18:27.515462 8278 ts_tablet_manager.cc:1939] T bf21e3f3d6f84065baeebbc9a9529200 P ef24a29e52744650bbe374199cd06f0f: Deleting consensus metadata
I20260805 23:18:27.517071 7713 catalog_manager.cc:4841] TS ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403): tablet bf21e3f3d6f84065baeebbc9a9529200 (table test-table [id=2606a329b4154df2bc4e0736c45a083f]) successfully deleted
I20260805 23:18:27.517297 7713 catalog_manager.cc:4841] TS d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037): tablet bf21e3f3d6f84065baeebbc9a9529200 (table test-table [id=2606a329b4154df2bc4e0736c45a083f]) successfully deleted
W20260805 23:18:27.522925 7712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:41377 (error 108)
I20260805 23:18:27.523039 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7947
I20260805 23:18:27.528120 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:27.527795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.0.0.1:55584)
0805 23:18:27.527944 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:27.528032 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260805 23:18:27.528362 7712 catalog_manager.cc:4584] TS 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf21e3f3d6f84065baeebbc9a9529200: Network error: Client connection negotiation failed: client connection to 127.23.88.65:41377: connect: Connection refused (error 111)
I20260805 23:18:27.532521 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8094
I20260805 23:18:27.539362 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 7694
I20260805 23:18:47.547922 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--webserver_interface=127.23.88.126
--webserver_port=42855
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:18:47.662779 8280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:47.663100 8280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:47.663143 8280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:47.667305 8280 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:18:47.667366 8280 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:18:47.667383 8280 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:18:47.667397 8280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:18:47.667416 8280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:47.667434 8280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:47.667449 8280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:18:47.667465 8280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:18:47.672741 8280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:45211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:47.673769 8280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:47.674834 8280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:47.687878 8289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:47.687878 8291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:47.687954 8288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:47.688246 8280 server_base.cc:1034] running on GCE node
I20260805 23:18:47.689141 8280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:47.690366 8280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:47.691573 8280 hybrid_clock.cc:648] HybridClock initialized: now 1785971927691561 us; error 34 us; skew 500 ppm
Aug 05 23:18:47 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971927, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:47.699831 8280 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:18:47.700876 8280 webserver.cc:460] Webserver started at http://127.23.88.126:42855/ using document root <none> and password file <none>
I20260805 23:18:47.701503 8280 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:47.701618 8280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:47.705717 8280 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:18:47.708137 8299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:47.709419 8280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:18:47.709583 8280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "ccafc7c8f879436c9c28401b920d285b"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "49e18cc45fd5b9808181317a6a493bc0"
server_key_iv: "7dbee7a8537a98ed91dbee165878cf37"
server_key_version: "encryptionkey@0"
I20260805 23:18:47.710187 8280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:47.737768 8280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:47.740871 8280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:47.741108 8280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:47.748302 8280 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:45211
I20260805 23:18:47.748661 8361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:45211 every 8 connection(s)
I20260805 23:18:47.749403 8280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:18:47.754098 8362 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:18:47.755373 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8280
I20260805 23:18:47.755810 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:41377
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=35507
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:47.755999 8362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap starting.
I20260805 23:18:47.774641 8362 log.cc:824] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:47.784148 8362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:47.784574 8362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap complete.
I20260805 23:18:47.789119 8362 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:47.789688 8362 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Initialized, Role: FOLLOWER
I20260805 23:18:47.790195 8362 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:47.790313 8362 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:18:47.790416 8362 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:18:47.790529 8362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:18:47.793040 8362 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:47.793386 8362 leader_election.cc:302] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccafc7c8f879436c9c28401b920d285b; no voters:
I20260805 23:18:47.793726 8362 leader_election.cc:288] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260805 23:18:47.793813 8366 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:18:47.794036 8366 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 LEADER]: Becoming Leader. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Running, Role: LEADER
I20260805 23:18:47.794440 8366 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:18:47.795163 8362 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:18:47.795583 8366 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccafc7c8f879436c9c28401b920d285b. Latest consensus state: current_term: 2 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:18:47.795696 8366 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:47.796118 8368 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:18:47.796278 8368 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:18:47.800948 8382 catalog_manager.cc:1259] Loaded cluster ID: 1f2f9eab49b8416ebdb3f8e8327eb454
I20260805 23:18:47.801033 8382 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: loading cluster ID for follower catalog manager: success
I20260805 23:18:47.803505 8382 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: acquiring CA information for follower catalog manager: success
I20260805 23:18:47.804955 8382 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:18:47.805083 8371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:18:47.807415 8371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:18:47.807875 8371 catalog_manager.cc:1259] Loaded cluster ID: 1f2f9eab49b8416ebdb3f8e8327eb454
I20260805 23:18:47.807931 8371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:18:47.808565 8371 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:18:47.809092 8371 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Loaded TSK: 0
I20260805 23:18:47.820708 8371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Generated new TSK 1
I20260805 23:18:47.820981 8371 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:18:47.876955 8364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:47.877228 8364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:47.877274 8364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:47.881484 8364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:47.881580 8364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:47.881677 8364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:18:47.887586 8364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:41377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=35507
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:47.888790 8364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:47.889866 8364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:47.901253 8392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:47.901293 8393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:47.901599 8364 server_base.cc:1034] running on GCE node
W20260805 23:18:47.901650 8395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:47.902137 8364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:47.902869 8364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:47.904055 8364 hybrid_clock.cc:648] HybridClock initialized: now 1785971927904026 us; error 47 us; skew 500 ppm
Aug 05 23:18:47 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971927, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:47.906970 8364 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:18:47.908087 8364 webserver.cc:460] Webserver started at http://127.23.88.65:35507/ using document root <none> and password file <none>
I20260805 23:18:47.908674 8364 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:47.908780 8364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:47.913213 8364 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:18:47.915619 8402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:47.917013 8364 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:18:47.917117 8364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "210606a0859c4995ac2213026d78cd67"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "374f01627c9af3fa85eebcaac0b9dfa7"
server_key_iv: "d02de52a852527a4da04bbfce58cdeba"
server_key_version: "encryptionkey@0"
I20260805 23:18:47.917727 8364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:47.936156 8364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:47.939610 8364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:47.939828 8364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:47.940521 8364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:47.941797 8410 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:18:47.945955 8364 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:18:47.946022 8364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260805 23:18:47.946074 8364 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:18:47.947789 8364 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:18:47.947851 8364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260805 23:18:47.948118 8410 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Bootstrap starting.
I20260805 23:18:47.959214 8364 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:41377
I20260805 23:18:47.959268 8526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:41377 every 8 connection(s)
I20260805 23:18:47.960508 8364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:18:47.963099 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8364
I20260805 23:18:47.963671 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:35403
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=44663
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:18:47.970803 8410 log.cc:824] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:18:47 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971927, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:47.980281 8530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:47.962801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:40365)
0805 23:18:47.964077 (+ 1276us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:47.964140 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:47.965408 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:47.966561 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:47.966568 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:47.967003 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:47.967531 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:47.967546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.968865 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.968868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:47.969806 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:47.969816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.970573 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.970576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:47.970770 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:47.971501 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:47.971519 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:47.973613 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:47.976594 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:47.976600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:47.976615 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:47.977013 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:47.977478 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:47.977481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:47.977483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:47.977654 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:47.978412 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:47.978418 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:47.978707 (+ 289us) client_negotiation.cc:769] Sending connection context
0805 23:18:47.978912 (+ 205us) client_negotiation.cc:240] Negotiation successful
0805 23:18:47.979278 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"thread_start_us":181,"threads_started":1}
I20260805 23:18:47.980792 8410 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:18:47.980903 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:47.962821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40365 (local address 127.23.88.126:45211)
0805 23:18:47.963499 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:47.963523 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:47.964305 (+ 782us) server_negotiation.cc:407] Connection header received
0805 23:18:47.965847 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:47.965858 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:47.966156 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:47.966500 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:47.967717 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.968674 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.969947 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.970414 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.970594 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:47.973866 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:47.973873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:47.973891 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:47.973928 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:47.976440 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:47.977170 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:47.977176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:47.977182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:47.977361 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:47.977778 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:47.977782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:47.977784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:47.978273 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:47.978434 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:47.979224 (+ 790us) server_negotiation.cc:299] Negotiation successful
0805 23:18:47.979564 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":137,"threads_started":1}
I20260805 23:18:47.981380 8410 tablet_bootstrap.cc:492] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Bootstrap complete.
I20260805 23:18:47.982908 8410 ts_tablet_manager.cc:1397] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Time spent bootstrapping tablet: real 0.035s user 0.025s sys 0.005s
I20260805 23:18:47.983641 8527 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:47.983879 8527 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:47.984470 8527 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:47.986269 8410 raft_consensus.cc:348] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:47.986866 8410 raft_consensus.cc:729] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210606a0859c4995ac2213026d78cd67, State: Initialized, Role: FOLLOWER
I20260805 23:18:47.987380 8410 consensus_queue.cc:260] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } }
I20260805 23:18:47.987475 8321 ts_manager.cc:194] Registered new tserver with Master: 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:47.988494 8410 ts_tablet_manager.cc:1428] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Time spent starting tablet: real 0.005s user 0.009s sys 0.000s
I20260805 23:18:47.991433 8321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:40365
I20260805 23:18:47.993916 8527 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
Aug 05 23:18:47 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971927, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:18:48.002159 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:47.990499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.0.0.1:44210)
0805 23:18:47.990904 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:47.990949 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:47.991074 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:47.991782 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:47.991788 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:47.992252 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:47.992400 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:47.992408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.993736 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.993740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:47.994976 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:47.994986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.995741 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.995744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:47.995915 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:47.996528 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:47.996539 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:47.998142 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:48.000365 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.000374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.000378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.000633 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.000979 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.000981 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.000993 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.001079 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.001607 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:48.001612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:48.001723 (+ 111us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.001850 (+ 127us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.002019 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":108,"threads_started":1}
I20260805 23:18:48.002327 8535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:47.990569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.23.88.65:41377)
0805 23:18:47.991267 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:47.991279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:47.991302 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:18:47.991370 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:47.991375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:47.991622 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:47.991757 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:47.992532 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.993615 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.995137 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:47.995634 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:47.995745 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:47.998294 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:47.998305 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:47.998316 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:47.998345 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:48.000226 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.000759 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.000765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.000767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.000865 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.001170 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.001174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.001176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.001484 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:48.001621 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.001965 (+ 344us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.002146 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":119,"threads_started":1}
I20260805 23:18:48.003409 8456 tablet_service.cc:1514] Processing DeleteTablet for tablet bf21e3f3d6f84065baeebbc9a9529200 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:27 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:44210
I20260805 23:18:48.004284 8537 tablet_replica.cc:330] stopping tablet replica
I20260805 23:18:48.004599 8537 raft_consensus.cc:2229] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:48.004861 8537 raft_consensus.cc:2258] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:48.005851 8537 ts_tablet_manager.cc:1905] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:48.009657 8537 ts_tablet_manager.cc:1918] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:48.009747 8537 log.cc:1196] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal/wals/bf21e3f3d6f84065baeebbc9a9529200
I20260805 23:18:48.010044 8537 ts_tablet_manager.cc:1939] T bf21e3f3d6f84065baeebbc9a9529200 P 210606a0859c4995ac2213026d78cd67: Deleting consensus metadata
I20260805 23:18:48.012080 8302 catalog_manager.cc:4841] TS 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377): tablet bf21e3f3d6f84065baeebbc9a9529200 (table test-table [id=2606a329b4154df2bc4e0736c45a083f]) successfully deleted
W20260805 23:18:48.087347 8531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:48.087647 8531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:48.087724 8531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:48.091820 8531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:48.091907 8531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:48.091991 8531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:18:48.097689 8531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:35403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=44663
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:48.098842 8531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:48.099963 8531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:48.110234 8543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:48.110235 8544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:48.110513 8531 server_base.cc:1034] running on GCE node
W20260805 23:18:48.110272 8546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:48.111052 8531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:48.111882 8531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:48.113057 8531 hybrid_clock.cc:648] HybridClock initialized: now 1785971928113024 us; error 45 us; skew 500 ppm
Aug 05 23:18:48 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971928, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:48.116441 8531 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:18:48.117450 8531 webserver.cc:460] Webserver started at http://127.23.88.66:44663/ using document root <none> and password file <none>
I20260805 23:18:48.118079 8531 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:48.118206 8531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:48.122659 8531 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:18:48.124895 8553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.126331 8531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:48.126502 8531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "ef24a29e52744650bbe374199cd06f0f"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "6f753962ead8361e82973020366d5297"
server_key_iv: "9d04a53fbd4e8f3181a8634368b61dd9"
server_key_version: "encryptionkey@0"
I20260805 23:18:48.127177 8531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:48.151554 8531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:48.155445 8531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:48.155629 8531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:48.156232 8531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:48.157099 8531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:48.157150 8531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.157195 8531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:48.157219 8531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.167068 8531 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35403
I20260805 23:18:48.167163 8678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35403 every 8 connection(s)
I20260805 23:18:48.168222 8531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:18:48.172639 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8531
I20260805 23:18:48.173210 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:39037
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=43863
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:18:48 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971928, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:48.184245 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.170129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:42489 (local address 127.23.88.126:45211)
0805 23:18:48.170382 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.170387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.171319 (+ 932us) server_negotiation.cc:407] Connection header received
0805 23:18:48.172768 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.172772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.172844 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.172939 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:48.174636 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.175659 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.176858 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.177114 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.177239 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.180224 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:48.180228 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:48.180230 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:48.180258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:48.181862 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.182517 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.182522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.182524 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.182610 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.182969 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.182972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.182974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.183278 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:48.183379 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.183928 (+ 549us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.184074 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260805 23:18:48.185182 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.170619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:42489)
0805 23:18:48.171148 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.171193 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.172488 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.173169 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.173181 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.173713 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:48.174434 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.174456 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.175805 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.175808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.176701 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.176711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.177290 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.177294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.177517 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.178187 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:48.178204 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:48.180024 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:48.182009 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.182015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.182032 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.182389 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.182707 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.182710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.182712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.182866 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.183394 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:48.183402 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:48.183669 (+ 267us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.183885 (+ 216us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.184245 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
I20260805 23:18:48.186591 8679 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:48.186865 8679 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:48.187443 8679 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:48.188640 8321 ts_manager.cc:194] Registered new tserver with Master: ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403)
I20260805 23:18:48.189429 8321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:42489
W20260805 23:18:48.288038 8682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:18:48.288306 8682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:18:48.288358 8682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:18:48.292366 8682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:18:48.292433 8682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:18:48.292492 8682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:18:48.298221 8682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:39037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=43863
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:18:48.299338 8682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:18:48.300307 8682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:18:48.310490 8690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:48.310469 8687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:18:48.310469 8688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:18:48.310684 8682 server_base.cc:1034] running on GCE node
I20260805 23:18:48.311429 8682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:18:48.312129 8682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:18:48.313338 8682 hybrid_clock.cc:648] HybridClock initialized: now 1785971928313303 us; error 48 us; skew 500 ppm
Aug 05 23:18:48 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971928, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:18:48.316372 8682 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:18:48.317281 8682 webserver.cc:460] Webserver started at http://127.23.88.67:43863/ using document root <none> and password file <none>
I20260805 23:18:48.317836 8682 fs_manager.cc:359] Metadata directory not provided
I20260805 23:18:48.317914 8682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:18:48.322467 8682 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:48.324709 8697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.326061 8682 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260805 23:18:48.326197 8682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "d1d2521331e74439bc69f26fec2b6194"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "a1f3d4e4c806b45327771da78fd13a31"
server_key_iv: "fe002d9a9db5eb9907b66c593b96437a"
server_key_version: "encryptionkey@0"
I20260805 23:18:48.326799 8682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:18:48.361317 8682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:18:48.364920 8682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:18:48.365144 8682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:18:48.365796 8682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:18:48.366650 8682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:18:48.366699 8682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.366777 8682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:18:48.366815 8682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:18:48.376435 8682 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:39037
I20260805 23:18:48.376471 8824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:39037 every 8 connection(s)
I20260805 23:18:48.377604 8682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:18:48.381278 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8682
Aug 05 23:18:48 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971928, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:18:48.391813 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.382237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.23.88.126:45211)
0805 23:18:48.382613 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.382617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.382642 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:18:48.382747 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.382750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.382797 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.382887 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:48.383875 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.384566 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.385747 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.386034 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.386166 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.388161 (+ 1995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:48.388164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:48.388166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:48.388190 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:48.389882 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.390459 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.390464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.390467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.390572 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.390928 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.390930 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.390931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.391171 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:48.391470 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.391541 (+ 71us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.391641 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":110,"threads_started":1}
I20260805 23:18:48.393829 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.379650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:42021 (local address 127.23.88.126:45211)
0805 23:18:48.379803 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.379808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.380806 (+ 998us) server_negotiation.cc:407] Connection header received
0805 23:18:48.382577 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.382581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.382642 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.382793 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:48.384318 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.384921 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.386324 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.386607 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.386720 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.389611 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:48.389615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:48.389617 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:48.389660 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:48.391412 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.392082 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.392086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.392091 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.392176 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.392584 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.392588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.392589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.392814 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:48.392909 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.393538 (+ 629us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.393677 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:18:48.394912 8828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.380048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:42021)
0805 23:18:48.380566 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.380613 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.382302 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.383037 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.383047 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.383590 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:18:48.384172 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.384186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.385052 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.385055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.386199 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.386209 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.386785 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.386792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.386998 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.387586 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:18:48.387610 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:18:48.389439 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:18:48.391552 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.391558 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.391574 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.391929 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.392286 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:18:48.392293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:18:48.392297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:18:48.392485 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:18:48.392958 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:18:48.392965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:18:48.393279 (+ 314us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.393495 (+ 216us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.393962 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":139,"threads_started":1}
I20260805 23:18:48.396159 8825 heartbeater.cc:347] Connected to a master server at 127.23.88.126:45211
I20260805 23:18:48.396445 8825 heartbeater.cc:464] Registering TS with master...
I20260805 23:18:48.396971 8825 heartbeater.cc:511] Master 127.23.88.126:45211 requested a full tablet report, sending...
I20260805 23:18:48.398074 8321 ts_manager.cc:194] Registered new tserver with Master: d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037)
I20260805 23:18:48.398663 8321 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:42021
I20260805 23:18:48.405421 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:18:48.409427 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.406045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37538 (local address 127.23.88.126:45211)
0805 23:18:48.406173 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.406184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.406417 (+ 233us) server_negotiation.cc:407] Connection header received
0805 23:18:48.406526 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.406528 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.406628 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.406711 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:48.406932 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.407576 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.408418 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.408624 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.408699 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.408969 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:48.409104 (+ 135us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0805 23:18:48.409307 (+ 203us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37538: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260805 23:18:48.409488 8529 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37538: token expired
W20260805 23:18:48.409806 8239 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.88.126:45211)
I20260805 23:18:48.417230 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.410187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37546 (local address 127.23.88.126:45211)
0805 23:18:48.410296 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.410300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.410316 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:18:48.410419 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.410421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.410459 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.410524 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:18:48.411268 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.411727 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.412482 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.412682 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.412760 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.414264 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:18:48.414267 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:18:48.414269 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:18:48.414292 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:18:48.415826 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.416238 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.416241 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.416242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.416319 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:18:48.416559 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:18:48.416561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:18:48.416562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:18:48.416726 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:18:48.416837 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.417009 (+ 172us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.417112 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:18:48.419854 8321 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:18:48.423501 8321 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:18:48.431746 8456 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e809c0c00412c947466809fab22ee (DEFAULT_TABLE table=test-table [id=db901647324d4251bac4815b86a7b8d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:48.432484 8456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e809c0c00412c947466809fab22ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:48.435500 8410 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Bootstrap starting.
I20260805 23:18:48.436416 8410 tablet_bootstrap.cc:654] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:48.436740 8534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.430613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35403 (local address 127.0.0.1:39636)
0805 23:18:48.430779 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.430798 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.430889 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.432051 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.432054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.432071 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:48.432215 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.432222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.433773 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.433777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.434889 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.434899 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.436278 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.436281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.436390 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.436438 (+ 48us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.436518 (+ 80us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.436608 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260805 23:18:48.437005 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.430869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39037 (local address 127.0.0.1:53110)
0805 23:18:48.431182 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.431197 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.431295 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.432612 (+ 1317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.432617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.432641 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:48.432752 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.432758 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.434229 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.434232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.435400 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.435414 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.436548 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.436552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.436665 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.436697 (+ 32us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.436755 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.436844 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":117,"threads_started":1}
I20260805 23:18:48.437135 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.431141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.23.88.66:35403)
0805 23:18:48.431511 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.431523 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.431547 (+ 24us) server_negotiation.cc:407] Connection header received
0805 23:18:48.431640 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.431646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.431870 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.432385 (+ 515us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:48.432566 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.433642 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.435041 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.436152 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.436279 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.436440 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.436625 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.436720 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":146,"threads_started":1}
I20260805 23:18:48.437772 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.431570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.23.88.67:39037)
0805 23:18:48.431915 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.431925 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.431967 (+ 42us) server_negotiation.cc:407] Connection header received
0805 23:18:48.432103 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.432111 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.432308 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.432909 (+ 601us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:48.433225 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.434094 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.435554 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.436396 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.437307 (+ 911us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.437452 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.437504 (+ 52us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.437597 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":114,"threads_started":1}
I20260805 23:18:48.438225 8410 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: No bootstrap required, opened a new log
I20260805 23:18:48.438319 8410 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260805 23:18:48.438766 8410 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.438896 8410 raft_consensus.cc:374] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:48.438944 8410 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210606a0859c4995ac2213026d78cd67, State: Initialized, Role: FOLLOWER
I20260805 23:18:48.439072 8410 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.439399 8410 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260805 23:18:48.440248 8753 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e809c0c00412c947466809fab22ee (DEFAULT_TABLE table=test-table [id=db901647324d4251bac4815b86a7b8d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:48.440248 8605 tablet_service.cc:1467] Processing CreateTablet for tablet 8e2e809c0c00412c947466809fab22ee (DEFAULT_TABLE table=test-table [id=db901647324d4251bac4815b86a7b8d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:18:48.441383 8605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e809c0c00412c947466809fab22ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:48.441401 8753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e809c0c00412c947466809fab22ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:18:48.449914 8841 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Bootstrap starting.
I20260805 23:18:48.450263 8840 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Bootstrap starting.
I20260805 23:18:48.452585 8841 tablet_bootstrap.cc:654] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:48.453118 8840 tablet_bootstrap.cc:654] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Neither blocks nor log segments found. Creating new log.
I20260805 23:18:48.454012 8840 log.cc:824] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:48.454013 8841 log.cc:824] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Log is configured to *not* fsync() on all Append() calls
I20260805 23:18:48.456318 8840 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: No bootstrap required, opened a new log
I20260805 23:18:48.456326 8841 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: No bootstrap required, opened a new log
I20260805 23:18:48.456550 8840 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:18:48.456555 8841 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:18:48.459522 8841 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.459522 8840 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.459774 8841 raft_consensus.cc:374] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:48.459801 8840 raft_consensus.cc:374] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:18:48.459832 8841 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d2521331e74439bc69f26fec2b6194, State: Initialized, Role: FOLLOWER
I20260805 23:18:48.459892 8840 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef24a29e52744650bbe374199cd06f0f, State: Initialized, Role: FOLLOWER
I20260805 23:18:48.460353 8840 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.460354 8841 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.461151 8679 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
I20260805 23:18:48.461378 8825 heartbeater.cc:503] Master 127.23.88.126:45211 was elected leader, sending a full tablet report...
I20260805 23:18:48.461498 8841 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260805 23:18:48.461874 8528 tablet.cc:2369] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:18:48.462042 8840 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:18:48.575006 8844 raft_consensus.cc:482] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:18:48.575335 8844 raft_consensus.cc:504] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.576633 8844 leader_election.cc:288] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037), 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:48.580900 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.577019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:45065 (local address 127.23.88.67:39037)
0805 23:18:48.577171 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.577177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.577316 (+ 139us) server_negotiation.cc:407] Connection header received
0805 23:18:48.577496 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.577502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.577559 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.577635 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:48.577952 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.578528 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.579818 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.580303 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.580400 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.580444 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.580687 (+ 243us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.580758 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:18:48.580924 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.576873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39037 (local address 127.23.88.66:45065)
0805 23:18:48.577211 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.577227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.577356 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.577660 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.577663 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.577681 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:48.577816 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.577827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.578662 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.578666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.579690 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.579700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.580424 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.580427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.580540 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.580567 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.580655 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.580744 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":24}
I20260805 23:18:48.581118 8846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.577005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.23.88.66:37381)
0805 23:18:48.577402 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.577443 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:18:48.577553 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:18:48.577895 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:18:48.577901 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:18:48.577919 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:18:48.578034 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.578041 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.578958 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.578960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.579898 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:18:48.579910 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.580702 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.580705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:18:48.580801 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.580824 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:18:48.580882 (+ 58us) client_negotiation.cc:240] Negotiation successful
0805 23:18:48.580955 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":24,"thread_start_us":129,"threads_started":1}
I20260805 23:18:48.581168 8847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.577116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:37381 (local address 127.23.88.65:41377)
0805 23:18:48.577440 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.577445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.577516 (+ 71us) server_negotiation.cc:407] Connection header received
0805 23:18:48.577659 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.577662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.577728 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.577855 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:18:48.578222 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.578833 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.580027 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.580574 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.580692 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.580855 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.580914 (+ 59us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.581020 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":112,"threads_started":1}
I20260805 23:18:48.581749 8478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "ef24a29e52744650bbe374199cd06f0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210606a0859c4995ac2213026d78cd67" is_pre_election: true
I20260805 23:18:48.581769 8775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "ef24a29e52744650bbe374199cd06f0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d2521331e74439bc69f26fec2b6194" is_pre_election: true
I20260805 23:18:48.582100 8478 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef24a29e52744650bbe374199cd06f0f in term 0.
I20260805 23:18:48.582098 8775 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef24a29e52744650bbe374199cd06f0f in term 0.
I20260805 23:18:48.582687 8555 leader_election.cc:302] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 210606a0859c4995ac2213026d78cd67, d1d2521331e74439bc69f26fec2b6194, ef24a29e52744650bbe374199cd06f0f; no voters:
I20260805 23:18:48.583010 8844 raft_consensus.cc:2781] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:18:48.583142 8844 raft_consensus.cc:482] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:18:48.583237 8844 raft_consensus.cc:3037] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:48.584326 8844 raft_consensus.cc:504] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.584765 8844 leader_election.cc:288] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [CANDIDATE]: Term 1 election: Requested vote from peers d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037), 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:18:48.585145 8775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "ef24a29e52744650bbe374199cd06f0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1d2521331e74439bc69f26fec2b6194"
I20260805 23:18:48.585196 8478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "ef24a29e52744650bbe374199cd06f0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "210606a0859c4995ac2213026d78cd67"
I20260805 23:18:48.585286 8775 raft_consensus.cc:3037] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:48.585363 8478 raft_consensus.cc:3037] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:18:48.586385 8775 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef24a29e52744650bbe374199cd06f0f in term 1.
I20260805 23:18:48.586459 8478 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef24a29e52744650bbe374199cd06f0f in term 1.
I20260805 23:18:48.586776 8556 leader_election.cc:302] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1d2521331e74439bc69f26fec2b6194, ef24a29e52744650bbe374199cd06f0f; no voters:
I20260805 23:18:48.586992 8844 raft_consensus.cc:2781] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:18:48.587252 8844 raft_consensus.cc:686] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 LEADER]: Becoming Leader. State: Replica: ef24a29e52744650bbe374199cd06f0f, State: Running, Role: LEADER
I20260805 23:18:48.587603 8844 consensus_queue.cc:237] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:18:48.591156 8320 catalog_manager.cc:5495] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f reported cstate change: term changed from 0 to 1, leader changed from <none> to ef24a29e52744650bbe374199cd06f0f (127.23.88.66). New cstate: current_term: 1 leader_uuid: "ef24a29e52744650bbe374199cd06f0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } health_report { overall_health: UNKNOWN } } }
W20260805 23:18:48.628659 8826 tablet.cc:2369] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:18:48.631413 8836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.627606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39648 (local address 127.23.88.66:35403)
0805 23:18:48.627748 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:18:48.627752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:18:48.627766 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:18:48.627872 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:18:48.627874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:18:48.627932 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:18:48.628037 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:18:48.628262 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.628747 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.629606 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:18:48.629894 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:18:48.630014 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:18:48.630330 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:18:48.630819 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:18:48.630965 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0805 23:18:48.631162 (+ 197us) server_negotiation.cc:299] Negotiation successful
0805 23:18:48.631229 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:18:48.637497 8478 raft_consensus.cc:1261] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Refusing update from remote peer ef24a29e52744650bbe374199cd06f0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:48.638185 8775 raft_consensus.cc:1261] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Refusing update from remote peer ef24a29e52744650bbe374199cd06f0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:18:48.638218 8844 consensus_queue.cc:1035] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:48.638788 8844 consensus_queue.cc:1035] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:18:48.648027 8853 mvcc.cc:205] Tried to move back new op lower bound from 7315341019691020288 to 7315341019497660416. Current Snapshot: MvccSnapshot[applied={T|T < 7315341019691020288}]
I20260805 23:18:48.655679 8320 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:48.655861 8320 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:18:48.658372 8320 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Sending DeleteTablet for 3 replicas of tablet 8e2e809c0c00412c947466809fab22ee
I20260805 23:18:48.659224 8456 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e809c0c00412c947466809fab22ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:48 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:44210
I20260805 23:18:48.659252 8605 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e809c0c00412c947466809fab22ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:48 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39636
I20260805 23:18:48.659492 8753 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e2e809c0c00412c947466809fab22ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:18:48 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:53110
I20260805 23:18:48.659871 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8364
I20260805 23:18:48.659888 8864 tablet_replica.cc:330] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: stopping tablet replica
I20260805 23:18:48.660002 8865 tablet_replica.cc:330] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: stopping tablet replica
I20260805 23:18:48.662483 8865 raft_consensus.cc:2229] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:18:48.662818 8865 raft_consensus.cc:2258] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:48.664424 8865 ts_tablet_manager.cc:1905] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:48.666651 8865 ts_tablet_manager.cc:1918] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:18:48.666724 8865 log.cc:1196] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/8e2e809c0c00412c947466809fab22ee
I20260805 23:18:48.666989 8865 ts_tablet_manager.cc:1939] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Deleting consensus metadata
I20260805 23:18:48.667608 8864 raft_consensus.cc:2229] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:18:48.668059 8864 raft_consensus.cc:2258] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:18:48.668512 8304 catalog_manager.cc:4841] TS d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037): tablet 8e2e809c0c00412c947466809fab22ee (table test-table [id=db901647324d4251bac4815b86a7b8d6]) successfully deleted
W20260805 23:18:48.669286 8302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:41377 (error 108)
I20260805 23:18:48.669772 8864 ts_tablet_manager.cc:1905] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:18:48.669929 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8531
I20260805 23:18:48.671465 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.671162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.0.0.1:44224)
0805 23:18:48.671326 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.671384 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:41377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260805 23:18:48.671617 8302 catalog_manager.cc:4584] TS 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e2e809c0c00412c947466809fab22ee: Network error: Client connection negotiation failed: client connection to 127.23.88.65:41377: connect: Connection refused (error 111)
I20260805 23:18:48.676452 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8682
I20260805 23:18:48.677027 8835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:18:48.676672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35403 (local address 127.0.0.1:39664)
0805 23:18:48.676868 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0805 23:18:48.676922 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:35403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260805 23:18:48.683343 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8280
I20260805 23:19:08.691205 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--webserver_interface=127.23.88.126
--webserver_port=42855
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:45211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:08.806160 8867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:08.806468 8867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:08.806511 8867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:08.811208 8867 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260805 23:19:08.811313 8867 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260805 23:19:08.811345 8867 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260805 23:19:08.811368 8867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:08.811400 8867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:08.811432 8867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:08.811467 8867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:08.811503 8867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:08.817603 8867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:45211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=42855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:08.818728 8867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:08.819770 8867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:19:08.829137 8867 server_base.cc:1034] running on GCE node
W20260805 23:19:08.829070 8872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:08.829053 8873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:08.829088 8875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:08.829902 8867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:08.831053 8867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:08.832296 8867 hybrid_clock.cc:648] HybridClock initialized: now 1785971948832285 us; error 27 us; skew 500 ppm
Aug 05 23:19:08 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971948, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:08.839040 8867 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:08.840154 8867 webserver.cc:460] Webserver started at http://127.23.88.126:42855/ using document root <none> and password file <none>
I20260805 23:19:08.840767 8867 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:08.840878 8867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:08.845160 8867 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:19:08.847586 8883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:08.848953 8867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:08.849107 8867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "ccafc7c8f879436c9c28401b920d285b"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "49e18cc45fd5b9808181317a6a493bc0"
server_key_iv: "7dbee7a8537a98ed91dbee165878cf37"
server_key_version: "encryptionkey@0"
I20260805 23:19:08.849707 8867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:08.864985 8867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:08.868649 8867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:08.868880 8867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:08.876844 8867 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:45211
I20260805 23:19:08.877179 8942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:45211 every 8 connection(s)
I20260805 23:19:08.878015 8867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:08.878767 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8867
I20260805 23:19:08.879565 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:41377
--local_ip_for_outbound_sockets=127.23.88.65
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=35507
--webserver_interface=127.23.88.65
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:19:08.883034 8943 sys_catalog.cc:263] Verifying existing consensus state
I20260805 23:19:08.884984 8943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap starting.
I20260805 23:19:08.903456 8943 log.cc:824] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:08.915040 8943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:19:08.915549 8943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Bootstrap complete.
I20260805 23:19:08.919962 8943 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:19:08.920473 8943 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Initialized, Role: FOLLOWER
I20260805 23:19:08.921015 8943 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:19:08.921175 8943 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:08.921264 8943 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:08.921433 8943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 2 FOLLOWER]: Advancing to term 3
I20260805 23:19:08.923581 8943 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:19:08.923930 8943 leader_election.cc:302] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccafc7c8f879436c9c28401b920d285b; no voters:
I20260805 23:19:08.924348 8943 leader_election.cc:288] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [CANDIDATE]: Term 3 election: Requested vote from peers
I20260805 23:19:08.924485 8947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 3 FOLLOWER]: Leader election won for term 3
I20260805 23:19:08.924690 8947 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [term 3 LEADER]: Becoming Leader. State: Replica: ccafc7c8f879436c9c28401b920d285b, State: Running, Role: LEADER
I20260805 23:19:08.925002 8947 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } }
I20260805 23:19:08.925680 8943 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:08.926018 8949 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccafc7c8f879436c9c28401b920d285b. Latest consensus state: current_term: 3 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:19:08.926151 8949 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:08.926450 8948 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ccafc7c8f879436c9c28401b920d285b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccafc7c8f879436c9c28401b920d285b" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45211 } } }
I20260805 23:19:08.926525 8951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:08.926549 8948 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:08.932060 8951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:08.932934 8951 catalog_manager.cc:1259] Loaded cluster ID: 1f2f9eab49b8416ebdb3f8e8327eb454
I20260805 23:19:08.933000 8951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:08.935676 8951 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:08.936463 8951 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Loaded TSK: 0
I20260805 23:19:08.936556 8951 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ccafc7c8f879436c9c28401b920d285b: Loaded TSK: 1
F20260805 23:19:08.937436 8951 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7916a209ad google::LogMessage::Fail() at ??:0
@ 0x7f7916a2388a google::LogMessage::SendToLog() at ??:0
@ 0x7f7916a204de google::LogMessage::Flush() at ??:0
@ 0x7f7916a21a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f79196e418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f79196dcf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f79213857bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f792135f99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f7921360178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f79213a1df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f79213c082b std::function<>::operator()() at ??:0
@ 0x7f792135fb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f7921360f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f792135d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f79213a1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7918c723d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7918c72c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f7918c743f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7918c63512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f791a2a76db start_thread at ??:0
@ 0x7f791551e61f clone at ??:0
W20260805 23:19:09.020900 8945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:09.021162 8945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:09.021207 8945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:09.025976 8945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:09.026027 8945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:09.026088 8945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:09.032073 8945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:41377
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=35507
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:09.033159 8945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:09.034179 8945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:09.044857 8990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.044839 8994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.044940 8991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:09.046084 8945 server_base.cc:1034] running on GCE node
I20260805 23:19:09.046689 8945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:09.047454 8945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:09.048642 8945 hybrid_clock.cc:648] HybridClock initialized: now 1785971949048602 us; error 52 us; skew 500 ppm
Aug 05 23:19:09 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:09.052044 8945 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:19:09.053406 8945 webserver.cc:460] Webserver started at http://127.23.88.65:35507/ using document root <none> and password file <none>
I20260805 23:19:09.054404 8945 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:09.054502 8945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:09.060473 8945 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:19:09.063414 9006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:09.065102 8945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:09.065253 8945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "210606a0859c4995ac2213026d78cd67"
format_stamp: "Formatted at 2026-08-05 23:18:26 on dist-test-slave-vkjp"
server_key: "374f01627c9af3fa85eebcaac0b9dfa7"
server_key_iv: "d02de52a852527a4da04bbfce58cdeba"
server_key_version: "encryptionkey@0"
I20260805 23:19:09.066037 8945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:09.085495 8945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:09.089552 8945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:09.089777 8945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:09.090606 8945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:09.092036 9014 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:19:09.096341 8945 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:19:09.096428 8945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260805 23:19:09.096496 8945 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:19:09.098843 8945 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:19:09.098910 8945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:19:09.099280 9014 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Bootstrap starting.
I20260805 23:19:09.110481 8945 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:41377
I20260805 23:19:09.110872 9129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:41377 every 8 connection(s)
I20260805 23:19:09.111845 8945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:09.117434 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 8945
I20260805 23:19:09.118096 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:35403
--local_ip_for_outbound_sockets=127.23.88.66
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=44663
--webserver_interface=127.23.88.66
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:19:09.120250 9014 log.cc:824] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:09.129556 9014 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:19:09.130043 9014 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Bootstrap complete.
I20260805 23:19:09.131608 9014 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260805 23:19:09.134974 9014 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:09.135589 9014 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210606a0859c4995ac2213026d78cd67, State: Initialized, Role: FOLLOWER
I20260805 23:19:09.136126 9014 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:09.137251 9014 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260805 23:19:09.205070 9133 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:19:09.114315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:42601)
0805 23:19:09.115005 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.115064 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:09.116350 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:09.204011 (+ 87661us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":164,"threads_started":1}
W20260805 23:19:09.205727 9130 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:45211 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:45211: Client connection negotiation failed: client connection to 127.23.88.126:45211: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260805 23:19:09.206672 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.206331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:56933)
0805 23:19:09.206508 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.206595 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260805 23:19:09.207525 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.207287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:42857)
0805 23:19:09.207408 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.207464 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:19:09.207748 9130 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260805 23:19:09.233700 9134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:09.233986 9134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:09.234032 9134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:09.238150 9134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:09.238222 9134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:09.238286 9134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:09.244055 9134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:35403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=44663
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:09.245121 9134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:09.246104 9134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:19:09.256198 9134 server_base.cc:1034] running on GCE node
W20260805 23:19:09.256162 9146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.256132 9144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.256124 9143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:09.256901 9134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:09.257578 9134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:09.258793 9134 hybrid_clock.cc:648] HybridClock initialized: now 1785971949258782 us; error 30 us; skew 500 ppm
Aug 05 23:19:09 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:09.262171 9134 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:19:09.263237 9134 webserver.cc:460] Webserver started at http://127.23.88.66:44663/ using document root <none> and password file <none>
I20260805 23:19:09.263824 9134 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:09.263922 9134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:09.268062 9134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260805 23:19:09.270411 9153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:09.271999 9134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:19:09.272159 9134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "ef24a29e52744650bbe374199cd06f0f"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "6f753962ead8361e82973020366d5297"
server_key_iv: "9d04a53fbd4e8f3181a8634368b61dd9"
server_key_version: "encryptionkey@0"
I20260805 23:19:09.272881 9134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:09.289397 9134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:09.293040 9134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:09.293277 9134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:09.293955 9134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:09.295213 9161 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260805 23:19:09.299805 9134 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260805 23:19:09.299942 9134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260805 23:19:09.300055 9134 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260805 23:19:09.301846 9134 ts_tablet_manager.cc:610] Registered 1 tablets
I20260805 23:19:09.301920 9134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260805 23:19:09.302342 9161 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Bootstrap starting.
I20260805 23:19:09.309415 9161 log.cc:824] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:09.312844 9134 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35403
I20260805 23:19:09.312973 9275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35403 every 8 connection(s)
I20260805 23:19:09.313777 9134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:09.316078 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9134
I20260805 23:19:09.316651 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:39037
--local_ip_for_outbound_sockets=127.23.88.67
--tserver_master_addrs=127.23.88.126:45211
--webserver_port=43863
--webserver_interface=127.23.88.67
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:19:09.318480 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.316055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:44253)
0805 23:19:09.317349 (+ 1294us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.317512 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1033,"thread_start_us":170,"threads_started":1}
W20260805 23:19:09.318941 9277 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:45211 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:45211: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
I20260805 23:19:09.319840 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.319505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:39041)
0805 23:19:09.319700 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.319765 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260805 23:19:09.320015 9161 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:19:09.320616 9161 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Bootstrap complete.
I20260805 23:19:09.320768 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.320432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:50889)
0805 23:19:09.320555 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.320604 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260805 23:19:09.320971 9277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:19:09.321105 9161 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Time spent bootstrapping tablet: real 0.019s user 0.018s sys 0.000s
I20260805 23:19:09.324375 9161 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:09.324885 9161 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef24a29e52744650bbe374199cd06f0f, State: Initialized, Role: FOLLOWER
I20260805 23:19:09.325397 9161 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:09.326635 9161 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:09.326752 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.326436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:46669)
0805 23:19:09.326588 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.326662 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260805 23:19:09.327625 9279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.327338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:47887)
0805 23:19:09.327472 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.327547 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260805 23:19:09.434015 9281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:09.434286 9281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:09.434332 9281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:09.438408 9281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:09.438481 9281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:09.438580 9281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:09.444180 9281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:39037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=43863
--tserver_master_addrs=127.23.88.126:45211
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:09.445287 9281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:09.446334 9281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:09.457185 9287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.457288 9288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:09.457302 9290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:09.457625 9281 server_base.cc:1034] running on GCE node
I20260805 23:19:09.458040 9281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:09.458707 9281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:09.459928 9281 hybrid_clock.cc:648] HybridClock initialized: now 1785971949459886 us; error 55 us; skew 500 ppm
Aug 05 23:19:09 dist-test-slave-vkjp krb5kdc[7681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:09.462971 9281 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:19:09.463999 9281 webserver.cc:460] Webserver started at http://127.23.88.67:43863/ using document root <none> and password file <none>
I20260805 23:19:09.464581 9281 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:09.464681 9281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:09.468588 9281 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:19:09.470849 9298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:09.472254 9281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:09.472394 9281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "d1d2521331e74439bc69f26fec2b6194"
format_stamp: "Formatted at 2026-08-05 23:18:27 on dist-test-slave-vkjp"
server_key: "a1f3d4e4c806b45327771da78fd13a31"
server_key_iv: "fe002d9a9db5eb9907b66c593b96437a"
server_key_version: "encryptionkey@0"
I20260805 23:19:09.473014 9281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:09.499825 9281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:09.503208 9281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:09.503428 9281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:09.504115 9281 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:09.505007 9281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:09.505084 9281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:09.505160 9281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:09.505204 9281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:09.514866 9281 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:39037
I20260805 23:19:09.514902 9417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:39037 every 8 connection(s)
I20260805 23:19:09.515923 9281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:09.516165 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9281
I20260805 23:19:09.519914 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.518073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:37499)
0805 23:19:09.518980 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.519172 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":550,"threads_started":1}
W20260805 23:19:09.520421 9418 heartbeater.cc:644] Failed to heartbeat to 127.23.88.126:45211 (0 consecutive failures): Network error: Failed to ping master at 127.23.88.126:45211: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
I20260805 23:19:09.521068 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.520807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:56203)
0805 23:19:09.520926 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.520976 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260805 23:19:09.521744 9422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:09.521543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:56843)
0805 23:19:09.521646 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:09.521690 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:19:09.521950 9418 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260805 23:19:10.209007 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.208498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:60665)
0805 23:19:10.208828 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.208896 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":93,"threads_started":1}
I20260805 23:19:10.329094 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.328450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:39785)
0805 23:19:10.328862 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.328946 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":122,"threads_started":1}
I20260805 23:19:10.492462 9425 raft_consensus.cc:482] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:10.492738 9425 raft_consensus.cc:504] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:10.494017 9425 leader_election.cc:288] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403), d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037)
Aug 05 23:19:10 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.67@KRBTEST.COM
Aug 05 23:19:10 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.66@KRBTEST.COM
I20260805 23:19:10.508281 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.494251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35403 (local address 127.23.88.65:56197)
0805 23:19:10.494441 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.494456 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:10.494587 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:10.496060 (+ 1473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:10.496067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:10.496569 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:10.497019 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:10.497034 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.498377 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.498386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:10.499473 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:10.499485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.500245 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.500249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:10.500441 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:10.501257 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:10.501291 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:10.503588 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:10.505650 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:10.505654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:10.505656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:10.505975 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:10.506342 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:10.506348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:10.506510 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:10.507238 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:10.507252 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:10.507500 (+ 248us) client_negotiation.cc:769] Sending connection context
0805 23:19:10.507696 (+ 196us) client_negotiation.cc:240] Negotiation successful
0805 23:19:10.508084 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":57,"spinlock_wait_cycles":27520}
I20260805 23:19:10.508306 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.494824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:44707 (local address 127.23.88.67:39037)
0805 23:19:10.495273 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:10.495296 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:10.495330 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:19:10.496030 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:10.496036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:10.496338 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:10.496631 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:10.497181 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.498263 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.499632 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.500153 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.500352 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:10.503426 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:10.503432 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:10.503445 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:10.503475 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:10.505478 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506105 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:10.506111 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:10.506115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:10.506237 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506622 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:10.506625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:10.506627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:10.507040 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:10.507264 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:10.507830 (+ 566us) server_negotiation.cc:299] Negotiation successful
0805 23:19:10.508111 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":154,"threads_started":1}
I20260805 23:19:10.508318 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.494504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:56197 (local address 127.23.88.66:35403)
0805 23:19:10.494906 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:10.494923 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:10.494977 (+ 54us) server_negotiation.cc:407] Connection header received
0805 23:19:10.495283 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:10.495288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:10.495666 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:10.495987 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:10.497210 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.498206 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.499609 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.500102 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.500306 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:10.503763 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:10.503769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:10.503781 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:10.503810 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:10.505524 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506101 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:10.506111 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:10.506115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:10.506237 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506611 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:10.506615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:10.506616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:10.507041 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:10.507294 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:10.507814 (+ 520us) server_negotiation.cc:299] Negotiation successful
0805 23:19:10.508096 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":139,"threads_started":1}
I20260805 23:19:10.508359 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.494619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39037 (local address 127.23.88.65:44707)
0805 23:19:10.494993 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.495034 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:10.495175 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:10.496569 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:10.496573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:10.496796 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:10.497022 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:10.497040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.498412 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.498418 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:10.499488 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:10.499509 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:10.500313 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:10.500318 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:10.500448 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:10.501305 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:10.501320 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:10.503173 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:10.505618 (+ 2445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:10.505625 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:10.505639 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:10.505970 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:10.506352 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:10.506357 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:10.506359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:10.506517 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:10.507238 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:10.507245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:10.507501 (+ 256us) client_negotiation.cc:769] Sending connection context
0805 23:19:10.507695 (+ 194us) client_negotiation.cc:240] Negotiation successful
0805 23:19:10.508174 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":372,"spinlock_wait_cycles":10496,"thread_start_us":68,"threads_started":1}
I20260805 23:19:10.510360 9228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "210606a0859c4995ac2213026d78cd67" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ef24a29e52744650bbe374199cd06f0f" is_pre_election: true
I20260805 23:19:10.510679 9228 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 210606a0859c4995ac2213026d78cd67 in term 1.
I20260805 23:19:10.511638 9009 leader_election.cc:302] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 210606a0859c4995ac2213026d78cd67, ef24a29e52744650bbe374199cd06f0f; no voters:
I20260805 23:19:10.511987 9425 raft_consensus.cc:2781] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260805 23:19:10.512079 9425 raft_consensus.cc:482] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260805 23:19:10.512030 9009 leader_election.cc:341] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037): Not found: Tablet not found: 8e2e809c0c00412c947466809fab22ee
I20260805 23:19:10.512133 9425 raft_consensus.cc:3037] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 1 FOLLOWER]: Advancing to term 2
I20260805 23:19:10.514624 9425 raft_consensus.cc:504] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:10.515019 9425 leader_election.cc:288] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 election: Requested vote from peers ef24a29e52744650bbe374199cd06f0f (127.23.88.66:35403), d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037)
I20260805 23:19:10.515527 9228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e2e809c0c00412c947466809fab22ee" candidate_uuid: "210606a0859c4995ac2213026d78cd67" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ef24a29e52744650bbe374199cd06f0f"
I20260805 23:19:10.515683 9228 raft_consensus.cc:3037] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 1 FOLLOWER]: Advancing to term 2
W20260805 23:19:10.515787 9009 leader_election.cc:341] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037): Not found: Tablet not found: 8e2e809c0c00412c947466809fab22ee
I20260805 23:19:10.518456 9228 raft_consensus.cc:2454] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 210606a0859c4995ac2213026d78cd67 in term 2.
I20260805 23:19:10.518832 9009 leader_election.cc:302] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 210606a0859c4995ac2213026d78cd67, ef24a29e52744650bbe374199cd06f0f; no voters: d1d2521331e74439bc69f26fec2b6194
I20260805 23:19:10.519022 9425 raft_consensus.cc:2781] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 2 FOLLOWER]: Leader election won for term 2
I20260805 23:19:10.519284 9425 raft_consensus.cc:686] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [term 2 LEADER]: Becoming Leader. State: Replica: 210606a0859c4995ac2213026d78cd67, State: Running, Role: LEADER
I20260805 23:19:10.519622 9425 consensus_queue.cc:237] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:10.520049 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.519755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:58585)
0805 23:19:10.519924 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.519974 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260805 23:19:10.522923 9431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.522463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:41521)
0805 23:19:10.522749 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.522824 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":70,"threads_started":1}
I20260805 23:19:10.973426 9228 raft_consensus.cc:1261] T 8e2e809c0c00412c947466809fab22ee P ef24a29e52744650bbe374199cd06f0f [term 2 FOLLOWER]: Refusing update from remote peer 210606a0859c4995ac2213026d78cd67: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260805 23:19:10.974334 9425 consensus_queue.cc:1035] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260805 23:19:10.976665 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.976170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:36875)
0805 23:19:10.976487 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.976562 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":87,"threads_started":1}
I20260805 23:19:10.979053 9427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.978743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:41617)
0805 23:19:10.978906 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.978968 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260805 23:19:10.979588 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:10.979285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:35247)
0805 23:19:10.979459 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:10.979515 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260805 23:19:10.979683 9009 consensus_peers.cc:487] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67 -> Peer d1d2521331e74439bc69f26fec2b6194 (127.23.88.67:39037): Couldn't send request to peer d1d2521331e74439bc69f26fec2b6194. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8e2e809c0c00412c947466809fab22ee. This is attempt 1: this message will repeat every 5th retry.
I20260805 23:19:11.401572 9439 ts_tablet_manager.cc:927] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Initiating tablet copy from peer 210606a0859c4995ac2213026d78cd67 (127.23.88.65:41377)
I20260805 23:19:11.402235 9439 tablet_copy_client.cc:316] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: tablet copy: Beginning tablet copy session from remote peer at address 127.23.88.65:41377
Aug 05 23:19:11 dist-test-slave-vkjp krb5kdc[7681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971949, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.65@KRBTEST.COM
I20260805 23:19:11.420902 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.402769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41377 (local address 127.23.88.67:57257)
0805 23:19:11.403067 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:11.403131 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:11.403273 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:11.403874 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:11.403880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:11.404284 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:11.404458 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:11.404466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:11.405508 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:11.405511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:11.406224 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:11.406234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:11.406922 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:11.406925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:11.407043 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:11.414699 (+ 7656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:11.414715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:11.416478 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:11.418869 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:11.418882 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:11.418887 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:11.419311 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:11.419663 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:11.419667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:11.419668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:11.419765 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:11.420261 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:11.420267 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:11.420436 (+ 169us) client_negotiation.cc:769] Sending connection context
0805 23:19:11.420568 (+ 132us) client_negotiation.cc:240] Negotiation successful
0805 23:19:11.420742 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":127,"threads_started":1}
I20260805 23:19:11.420953 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.402980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:57257 (local address 127.23.88.65:41377)
0805 23:19:11.403404 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:11.403410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:11.403435 (+ 25us) server_negotiation.cc:407] Connection header received
0805 23:19:11.403488 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:11.403493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:11.403676 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:11.403826 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:11.404613 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:11.405387 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:11.406370 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:11.406772 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:11.406885 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:11.416723 (+ 9838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:11.416739 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:11.416754 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:11.416788 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:11.418692 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:11.419456 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:11.419463 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:11.419465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:11.419551 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:11.419863 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:11.419865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:11.419866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:11.420147 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:11.420250 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:11.420670 (+ 420us) server_negotiation.cc:299] Negotiation successful
0805 23:19:11.420806 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":63,"threads_started":1}
I20260805 23:19:11.422230 9102 tablet_copy_service.cc:140] P 210606a0859c4995ac2213026d78cd67: Received BeginTabletCopySession request for tablet 8e2e809c0c00412c947466809fab22ee from peer d1d2521331e74439bc69f26fec2b6194 ({username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:57257)
I20260805 23:19:11.422410 9102 tablet_copy_service.cc:161] P 210606a0859c4995ac2213026d78cd67: Beginning new tablet copy session on tablet 8e2e809c0c00412c947466809fab22ee from peer d1d2521331e74439bc69f26fec2b6194 at {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:57257: session id = d1d2521331e74439bc69f26fec2b6194-8e2e809c0c00412c947466809fab22ee
I20260805 23:19:11.423724 9102 tablet_copy_source_session.cc:205] T 8e2e809c0c00412c947466809fab22ee P 210606a0859c4995ac2213026d78cd67: Tablet Copy: opened 0 blocks and 1 log segments
I20260805 23:19:11.427965 9439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e2e809c0c00412c947466809fab22ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:11.434638 9439 tablet_copy_client.cc:791] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: tablet copy: Starting download of 0 data blocks...
I20260805 23:19:11.434849 9439 tablet_copy_client.cc:655] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: tablet copy: Starting download of 1 WAL segments...
I20260805 23:19:11.436820 9439 tablet_copy_client.cc:531] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260805 23:19:11.438810 9439 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Bootstrap starting.
I20260805 23:19:11.444599 9439 log.cc:824] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:11.451522 9439 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260805 23:19:11.451838 9439 tablet_bootstrap.cc:492] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Bootstrap complete.
I20260805 23:19:11.452142 9439 ts_tablet_manager.cc:1397] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Time spent bootstrapping tablet: real 0.014s user 0.014s sys 0.000s
I20260805 23:19:11.454854 9439 raft_consensus.cc:348] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:11.455276 9439 raft_consensus.cc:729] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1d2521331e74439bc69f26fec2b6194, State: Initialized, Role: FOLLOWER
I20260805 23:19:11.455948 9439 consensus_queue.cc:260] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef24a29e52744650bbe374199cd06f0f" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35403 } } peers { permanent_uuid: "d1d2521331e74439bc69f26fec2b6194" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39037 } } peers { permanent_uuid: "210606a0859c4995ac2213026d78cd67" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41377 } }
I20260805 23:19:11.457032 9439 ts_tablet_manager.cc:1428] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:11.457288 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.457017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:59141)
0805 23:19:11.457153 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:11.457212 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260805 23:19:11.457808 9102 tablet_copy_service.cc:342] P 210606a0859c4995ac2213026d78cd67: Request end of tablet copy session d1d2521331e74439bc69f26fec2b6194-8e2e809c0c00412c947466809fab22ee received from {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:57257
I20260805 23:19:11.457945 9102 tablet_copy_service.cc:433] P 210606a0859c4995ac2213026d78cd67: ending tablet copy session d1d2521331e74439bc69f26fec2b6194-8e2e809c0c00412c947466809fab22ee on tablet 8e2e809c0c00412c947466809fab22ee with peer d1d2521331e74439bc69f26fec2b6194
I20260805 23:19:11.458102 9441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.457838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:55173)
0805 23:19:11.457970 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:11.458025 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260805 23:19:11.980365 9447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.979885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:45033)
0805 23:19:11.980194 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:11.980256 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260805 23:19:11.980818 9448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:11.980358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:46515)
0805 23:19:11.980661 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:11.980719 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":83,"threads_started":1}
I20260805 23:19:12.017236 9447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:12.016926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:51911)
0805 23:19:12.017091 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:12.017152 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 23:19:12.017484 9370 raft_consensus.cc:1203] T 8e2e809c0c00412c947466809fab22ee P d1d2521331e74439bc69f26fec2b6194 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260805 23:19:12.018791 9452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:12.018353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:52045)
0805 23:19:12.018599 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:12.018687 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
I20260805 23:19:12.982182 9453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:12.981615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:44911)
0805 23:19:12.982009 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:12.982080 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":77,"threads_started":1}
I20260805 23:19:13.018738 9454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:13.018210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:39925)
0805 23:19:13.018550 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:13.018633 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":83,"threads_started":1}
I20260805 23:19:13.020167 9455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:13.019738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:35537)
0805 23:19:13.020012 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:13.020073 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":78,"threads_started":1}
I20260805 23:19:13.983733 9456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:13.983218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:35097)
0805 23:19:13.983501 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:13.983569 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260805 23:19:14.020300 9457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:14.019818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:54401)
0805 23:19:14.020130 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:14.020196 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
I20260805 23:19:14.021476 9458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:14.021018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:47757)
0805 23:19:14.021331 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:14.021390 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":67,"threads_started":1}
I20260805 23:19:14.985251 9460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:14.984774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:47509)
0805 23:19:14.985077 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:14.985145 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260805 23:19:15.022002 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:15.021441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:59465)
0805 23:19:15.021738 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:15.021873 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":78,"threads_started":1}
I20260805 23:19:15.022915 9461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:15.022622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:53657)
0805 23:19:15.022765 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:15.022828 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260805 23:19:15.022924 9462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:15.022484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:39289)
0805 23:19:15.022782 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:15.022834 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":61,"threads_started":1}
I20260805 23:19:15.987155 9463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:15.986645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:41733)
0805 23:19:15.986933 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:15.986987 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260805 23:19:16.024349 9464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:16.023843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:54161)
0805 23:19:16.024180 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:16.024244 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":103,"threads_started":1}
I20260805 23:19:16.024349 9465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:16.023909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:33933)
0805 23:19:16.024200 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:16.024257 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":104,"threads_started":1}
I20260805 23:19:16.988574 9466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:16.988079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:35687)
0805 23:19:16.988396 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:16.988468 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":84,"threads_started":1}
I20260805 23:19:17.026018 9467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:17.025399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:33049)
0805 23:19:17.025856 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:17.025913 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":100,"threads_started":1}
I20260805 23:19:17.026427 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:17.025407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:57029)
0805 23:19:17.026282 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:17.026340 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":771,"thread_start_us":94,"threads_started":1}
I20260805 23:19:17.990095 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:17.989573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:46389)
0805 23:19:17.989914 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:17.989986 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
I20260805 23:19:18.027603 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:18.027033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:41993)
0805 23:19:18.027416 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:18.027484 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":174,"threads_started":1}
I20260805 23:19:18.027788 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:18.027295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:36449)
0805 23:19:18.027577 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:18.027641 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":111,"threads_started":1}
I20260805 23:19:18.991618 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:18.991075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:49303)
0805 23:19:18.991453 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:18.991517 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":101,"threads_started":1}
I20260805 23:19:19.029074 9475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:19.028581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:44599)
0805 23:19:19.028896 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:19.028965 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
I20260805 23:19:19.029145 9474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:19.028586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:45607)
0805 23:19:19.028987 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:19.029047 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":105,"threads_started":1}
I20260805 23:19:19.993314 9476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:19.992793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:42595)
0805 23:19:19.993141 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:19.993211 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":95,"threads_started":1}
I20260805 23:19:20.030609 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:20.030119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:49077)
0805 23:19:20.030435 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:20.030507 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":99,"threads_started":1}
I20260805 23:19:20.030665 9478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:20.030118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:41735)
0805 23:19:20.030512 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:20.030571 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":102,"threads_started":1}
I20260805 23:19:20.994831 9479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:20.994341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:54397)
0805 23:19:20.994662 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:20.994729 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":95,"threads_started":1}
I20260805 23:19:21.031960 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:21.031515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:36577)
0805 23:19:21.031802 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:21.031860 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
I20260805 23:19:21.031960 9481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:21.031561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:60955)
0805 23:19:21.031811 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:21.031870 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":104,"threads_started":1}
I20260805 23:19:21.996119 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:21.995665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:41509)
0805 23:19:21.995928 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:21.995999 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":81,"threads_started":1}
I20260805 23:19:22.033406 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:22.032895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:48447)
0805 23:19:22.033235 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:22.033302 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":119,"threads_started":1}
I20260805 23:19:22.033456 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:22.032957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:45285)
0805 23:19:22.033261 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:22.033362 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
I20260805 23:19:22.997476 9485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:22.996972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:54069)
0805 23:19:22.997279 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:22.997376 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":105,"threads_started":1}
I20260805 23:19:23.034957 9486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:23.034335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:38835)
0805 23:19:23.034751 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:23.034845 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":94,"threads_started":1}
I20260805 23:19:23.034996 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:23.034362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:33511)
0805 23:19:23.034814 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:23.034886 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":101,"threads_started":1}
I20260805 23:19:23.998799 9488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:23.998343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.66:51463)
0805 23:19:23.998632 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:23.998700 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260805 23:19:24.036444 9490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:24.035934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.67:41621)
0805 23:19:24.036256 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:24.036339 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260805 23:19:24.036588 9489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:24.035934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:45211 (local address 127.23.88.65:51027)
0805 23:19:24.036380 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:24.036454 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.126:45211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":103,"threads_started":1}
I20260805 23:19:24.518834 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 8945
I20260805 23:19:24.528283 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9134
I20260805 23:19:24.535324 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9281
2026-08-05T23:19:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59348 ms)
[----------] 30 tests from SecurityITest (248796 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:24 dist-test-slave-vkjp krb5kdc[9493](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:24 dist-test-slave-vkjp krb5kdc[9493](info): set up 2 sockets
Aug 05 23:19:24 dist-test-slave-vkjp krb5kdc[9493](info): commencing operation
krb5kdc: starting...
W20260805 23:19:25.629240 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:25 dist-test-slave-vkjp krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:25.767279 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41123
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:25.884584 9509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:25.884924 9509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:25.885007 9509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:25.889259 9509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:25.889381 9509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:25.889420 9509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:25.889467 9509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:25.889503 9509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:25.895226 9509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:25.896783 9509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:25.897990 9509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:25.907656 9517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:25.907624 9515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:25.907678 9509 server_base.cc:1034] running on GCE node
W20260805 23:19:25.907624 9514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:25.908540 9509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:25.909715 9509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:25.910913 9509 hybrid_clock.cc:648] HybridClock initialized: now 1785971965910895 us; error 37 us; skew 500 ppm
Aug 05 23:19:25 dist-test-slave-vkjp krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971965, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:25.918989 9509 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:25.920166 9509 webserver.cc:460] Webserver started at http://127.23.88.126:37097/ using document root <none> and password file <none>
I20260805 23:19:25.920835 9509 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:25.920907 9509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:25.921188 9509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:25.923627 9509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "23992a2df7894f8cae1d78257ce1993a"
format_stamp: "Formatted at 2026-08-05 23:19:25 on dist-test-slave-vkjp"
server_key: "cfefbf9c7bcfc0858e779bbed8f5b1ba"
server_key_iv: "6c016c781c1b4c216dee899386953925"
server_key_version: "encryptionkey@0"
I20260805 23:19:25.924279 9509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "23992a2df7894f8cae1d78257ce1993a"
format_stamp: "Formatted at 2026-08-05 23:19:25 on dist-test-slave-vkjp"
server_key: "cfefbf9c7bcfc0858e779bbed8f5b1ba"
server_key_iv: "6c016c781c1b4c216dee899386953925"
server_key_version: "encryptionkey@0"
I20260805 23:19:25.928514 9509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:25.931365 9525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:25.932802 9509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:25.932991 9509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "23992a2df7894f8cae1d78257ce1993a"
format_stamp: "Formatted at 2026-08-05 23:19:25 on dist-test-slave-vkjp"
server_key: "cfefbf9c7bcfc0858e779bbed8f5b1ba"
server_key_iv: "6c016c781c1b4c216dee899386953925"
server_key_version: "encryptionkey@0"
I20260805 23:19:25.933128 9509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:25.948422 9509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:25.951854 9509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:25.952080 9509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:25.959830 9509 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41123
I20260805 23:19:25.959863 9579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41123 every 8 connection(s)
I20260805 23:19:25.960824 9509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:25.963639 9580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:25.964293 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9509
I20260805 23:19:25.964442 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:25.964740 23905 external_mini_cluster.cc:1428] Setting key e5c595b651e5eaafa45db194f2df9b90
I20260805 23:19:25.970435 9580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: Bootstrap starting.
I20260805 23:19:25.973169 9580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:25.973917 9580 log.cc:824] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: Log is configured to *not* fsync() on all Append() calls
Aug 05 23:19:25 dist-test-slave-vkjp krb5kdc[9493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:25.976415 9580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: No bootstrap required, opened a new log
I20260805 23:19:25.980358 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:25.966153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48148 (local address 127.23.88.126:41123)
0805 23:19:25.966588 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:25.966608 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:25.966653 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:19:25.967515 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:25.967538 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:25.967960 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:25.968313 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:25.969010 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:25.970477 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:25.971424 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:25.972140 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:25.972292 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:25.975026 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:25.975032 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:25.975048 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:25.975095 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:25.977017 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:25.977475 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:25.977481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:25.977487 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:25.977621 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:25.977990 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:25.977993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:25.977995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:25.978557 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:25.978788 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:25.979045 (+ 257us) server_negotiation.cc:299] Negotiation successful
0805 23:19:25.979377 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260805 23:19:25.981087 9580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } }
I20260805 23:19:25.981371 9580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:25.981426 9580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23992a2df7894f8cae1d78257ce1993a, State: Initialized, Role: FOLLOWER
I20260805 23:19:25.981894 9580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } }
I20260805 23:19:25.982023 9580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:25.982106 9580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:25.982211 9580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:25.983422 9580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } }
I20260805 23:19:25.983793 9580 leader_election.cc:302] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23992a2df7894f8cae1d78257ce1993a; no voters:
I20260805 23:19:25.984108 9580 leader_election.cc:288] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:25.984259 9585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:25.984501 9585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [term 1 LEADER]: Becoming Leader. State: Replica: 23992a2df7894f8cae1d78257ce1993a, State: Running, Role: LEADER
I20260805 23:19:25.984850 9585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } }
I20260805 23:19:25.985332 9580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:25.986572 9587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23992a2df7894f8cae1d78257ce1993a. Latest consensus state: current_term: 1 leader_uuid: "23992a2df7894f8cae1d78257ce1993a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } } }
I20260805 23:19:25.986711 9587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:25.987288 9595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:25.987267 9586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23992a2df7894f8cae1d78257ce1993a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23992a2df7894f8cae1d78257ce1993a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41123 } } }
I20260805 23:19:25.987507 9586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:25.990171 9595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:25.995510 9595 catalog_manager.cc:1347] Generated new cluster ID: e7194823b7d94884a02623f9243c8e4b
I20260805 23:19:25.995585 9595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:26.004240 9595 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:26.005515 9595 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:26.012629 9595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: Generated new TSK 0
I20260805 23:19:26.013553 9595 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:19:26.046448 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:26.163735 9607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:26.164017 9607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:26.164062 9607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:26.168306 9607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:26.168375 9607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:26.168488 9607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:26.174054 9607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:26.175284 9607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:26.176339 9607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:26.186417 9613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:26.186482 9612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:26.186437 9615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:26.186717 9607 server_base.cc:1034] running on GCE node
I20260805 23:19:26.187341 9607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:26.188087 9607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:26.189273 9607 hybrid_clock.cc:648] HybridClock initialized: now 1785971966189248 us; error 47 us; skew 500 ppm
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:26.192184 9607 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:19:26.193114 9607 webserver.cc:460] Webserver started at http://127.23.88.65:37855/ using document root <none> and password file <none>
I20260805 23:19:26.193717 9607 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:26.193815 9607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:26.194103 9607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:26.196290 9607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "4e919ce3472a428a8aaa8e01eb1849ca"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "d2807e0acd8bc729a088c9cd87a9c2fc"
server_key_iv: "a6b5cdba07c5a483660ac18473acba9d"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.196889 9607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "4e919ce3472a428a8aaa8e01eb1849ca"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "d2807e0acd8bc729a088c9cd87a9c2fc"
server_key_iv: "a6b5cdba07c5a483660ac18473acba9d"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.200927 9607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:26.203866 9622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.205191 9607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:26.205312 9607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "4e919ce3472a428a8aaa8e01eb1849ca"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "d2807e0acd8bc729a088c9cd87a9c2fc"
server_key_iv: "a6b5cdba07c5a483660ac18473acba9d"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.205415 9607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:26.226768 9607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:26.230165 9607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:26.230340 9607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:26.230916 9607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:26.231817 9607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:26.231865 9607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.231910 9607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:26.231926 9607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.241111 9607 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:39519
I20260805 23:19:26.241137 9744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:39519 every 8 connection(s)
I20260805 23:19:26.242035 9607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:26.243767 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9607
I20260805 23:19:26.243928 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:26.244267 23905 external_mini_cluster.cc:1428] Setting key f8aa5420e7a1ed038aa2e3e7ad83e8d6
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:19:26.258260 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.244048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:57443 (local address 127.23.88.126:41123)
0805 23:19:26.244175 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.244178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.245250 (+ 1072us) server_negotiation.cc:407] Connection header received
0805 23:19:26.246723 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.246726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.246781 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.246860 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:26.248574 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.249150 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.251001 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.251301 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.251398 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.254355 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:26.254359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:26.254362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:26.254395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:26.256074 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.256614 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.256618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.256620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.256701 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.257054 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.257058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.257060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.257265 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:26.257374 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.257927 (+ 553us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.258076 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260805 23:19:26.259166 9747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.244538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41123 (local address 127.23.88.65:57443)
0805 23:19:26.245096 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.245156 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.246513 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.247032 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.247039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.247656 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:26.248383 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.248415 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.249294 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.249301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.250853 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.250868 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.251486 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.251494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.251748 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.252445 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:26.252473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:26.254173 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:26.256201 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.256207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.256223 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.256499 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.256798 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.256802 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.256804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.256955 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.257380 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:26.257389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:26.257653 (+ 264us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.257885 (+ 232us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.258228 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":187,"threads_started":1}
I20260805 23:19:26.260681 9745 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41123
I20260805 23:19:26.260946 9745 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:26.261574 9745 heartbeater.cc:511] Master 127.23.88.126:41123 requested a full tablet report, sending...
I20260805 23:19:26.263255 9543 ts_manager.cc:194] Registered new tserver with Master: 4e919ce3472a428a8aaa8e01eb1849ca (127.23.88.65:39519)
I20260805 23:19:26.264447 9543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:57443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:19:26.275213 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:26.394425 9750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:26.394693 9750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:26.394764 9750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:26.398944 9750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:26.399014 9750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:26.399076 9750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:26.404762 9750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:26.405900 9750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:26.406988 9750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:26.417281 9758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:26.417251 9755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:26.417273 9756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:26.417687 9750 server_base.cc:1034] running on GCE node
I20260805 23:19:26.418174 9750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:26.418902 9750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:26.420094 9750 hybrid_clock.cc:648] HybridClock initialized: now 1785971966420061 us; error 51 us; skew 500 ppm
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:26.423220 9750 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:19:26.424258 9750 webserver.cc:460] Webserver started at http://127.23.88.66:45781/ using document root <none> and password file <none>
I20260805 23:19:26.424850 9750 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:26.424942 9750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:26.425158 9750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:26.427357 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "b22277cf57934680ba40666d79e2bff4"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "7f4129d2942436e6ee16e9c176504784"
server_key_iv: "a68f3377ddce124678280bbc376fd95a"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.427937 9750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "b22277cf57934680ba40666d79e2bff4"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "7f4129d2942436e6ee16e9c176504784"
server_key_iv: "a68f3377ddce124678280bbc376fd95a"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.431879 9750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:19:26.434420 9765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.435807 9750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:19:26.435945 9750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "b22277cf57934680ba40666d79e2bff4"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "7f4129d2942436e6ee16e9c176504784"
server_key_iv: "a68f3377ddce124678280bbc376fd95a"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.436059 9750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:26.467278 9750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:26.470844 9750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:26.471112 9750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:26.471866 9750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:26.472752 9750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:26.472803 9750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.472844 9750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:26.472859 9750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.482290 9750 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:37409
I20260805 23:19:26.482322 9886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:37409 every 8 connection(s)
I20260805 23:19:26.483353 9750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:26.493024 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9750
I20260805 23:19:26.493170 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:26.493460 23905 external_mini_cluster.cc:1428] Setting key 556b03f8be0e1cccc43cc3eb5c7a6dae
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:26.497910 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.485285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:35389 (local address 127.23.88.126:41123)
0805 23:19:26.485470 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.485475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.486387 (+ 912us) server_negotiation.cc:407] Connection header received
0805 23:19:26.487969 (+ 1582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.487975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.488070 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.488148 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:26.489303 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.489865 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.491029 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.491416 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.491523 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.494178 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:26.494184 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:26.494186 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:26.494220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:26.495763 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.496328 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.496332 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.496334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.496418 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.496764 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.496768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.496770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.496970 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:26.497081 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.497599 (+ 518us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.497730 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260805 23:19:26.498761 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.485677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41123 (local address 127.23.88.66:35389)
0805 23:19:26.486205 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.486250 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.487728 (+ 1478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.488323 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.488330 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.488681 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:26.489162 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.489175 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.489986 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.489988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.490880 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.490890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.491567 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.491570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.491710 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.492267 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:26.492284 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:26.494002 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:26.495907 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.495913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.495929 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.496215 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.496516 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.496519 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.496521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.496668 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.497083 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:26.497092 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:26.497354 (+ 262us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.497560 (+ 206us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.497900 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
I20260805 23:19:26.500005 9887 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41123
I20260805 23:19:26.500295 9887 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:26.500970 9887 heartbeater.cc:511] Master 127.23.88.126:41123 requested a full tablet report, sending...
I20260805 23:19:26.502255 9543 ts_manager.cc:194] Registered new tserver with Master: b22277cf57934680ba40666d79e2bff4 (127.23.88.66:37409)
I20260805 23:19:26.503186 9543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:35389
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:19:26.524250 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:26.636355 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:26.636641 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:26.636693 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:26.641126 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:26.641194 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:26.641285 9893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:26.646754 9893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:26.647914 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:26.648978 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:26.658674 9902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:26.658733 9899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:26.658774 9893 server_base.cc:1034] running on GCE node
W20260805 23:19:26.658993 9900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:26.659415 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:26.660142 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:26.661340 9893 hybrid_clock.cc:648] HybridClock initialized: now 1785971966661320 us; error 37 us; skew 500 ppm
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:26.664523 9893 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:19:26.665548 9893 webserver.cc:460] Webserver started at http://127.23.88.67:42425/ using document root <none> and password file <none>
I20260805 23:19:26.666136 9893 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:26.666199 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:26.666422 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:26.668648 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "a14dea609d9743909d3bdce1669a5db3"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "1dcd8a67a00f87d5efb41993e905361c"
server_key_iv: "11a8e646b61a2130b83cce4d9f953289"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.669230 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "a14dea609d9743909d3bdce1669a5db3"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "1dcd8a67a00f87d5efb41993e905361c"
server_key_iv: "11a8e646b61a2130b83cce4d9f953289"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.673143 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:26.675697 9910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.676955 9893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:26.677093 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "a14dea609d9743909d3bdce1669a5db3"
format_stamp: "Formatted at 2026-08-05 23:19:26 on dist-test-slave-vkjp"
server_key: "1dcd8a67a00f87d5efb41993e905361c"
server_key_iv: "11a8e646b61a2130b83cce4d9f953289"
server_key_version: "encryptionkey@0"
I20260805 23:19:26.677208 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:26.691833 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:26.695156 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:26.695385 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:26.696033 9893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:26.696943 9893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:26.696993 9893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.697058 9893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:26.697093 9893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:26.707237 9893 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:40437
I20260805 23:19:26.707281 10030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:40437 every 8 connection(s)
I20260805 23:19:26.708199 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:26.711055 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 9893
I20260805 23:19:26.711241 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:26.711587 23905 external_mini_cluster.cc:1428] Setting key 37e7a04d8a25adffc59e33b9c32f1c36
Aug 05 23:19:26 dist-test-slave-vkjp krb5kdc[9493](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971966, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:26.722522 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.710179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:53729 (local address 127.23.88.126:41123)
0805 23:19:26.710318 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.710322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.711286 (+ 964us) server_negotiation.cc:407] Connection header received
0805 23:19:26.712934 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.712936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.713013 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.713104 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:26.714480 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.715010 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.716281 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.716571 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.716654 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.719292 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:26.719295 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:26.719297 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:26.719323 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:26.720596 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.721109 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.721112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.721113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.721186 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.721509 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.721511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.721512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.721689 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:26.721765 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.722277 (+ 512us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.722385 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:19:26.723426 10033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.710520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41123 (local address 127.23.88.67:53729)
0805 23:19:26.711074 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.711167 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.712689 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.713307 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.713315 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.713707 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:26.714322 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.714336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.715167 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.715169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.716158 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.716168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.716708 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.716710 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.716850 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.717517 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:26.717534 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:26.719033 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:26.720704 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.720709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.720723 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.721013 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.721268 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:26.721271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:26.721272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:26.721410 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:26.721777 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:26.721782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:26.722041 (+ 259us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.722196 (+ 155us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.722468 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":116,"threads_started":1}
I20260805 23:19:26.724630 10031 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41123
I20260805 23:19:26.724885 10031 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:26.725387 10031 heartbeater.cc:511] Master 127.23.88.126:41123 requested a full tablet report, sending...
I20260805 23:19:26.726398 9543 ts_manager.cc:194] Registered new tserver with Master: a14dea609d9743909d3bdce1669a5db3 (127.23.88.67:40437)
I20260805 23:19:26.726981 9543 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:53729
I20260805 23:19:26.737854 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:26.747253 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.739931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48158 (local address 127.23.88.126:41123)
0805 23:19:26.740090 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.740097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.740251 (+ 154us) server_negotiation.cc:407] Connection header received
0805 23:19:26.740353 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.740355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.740397 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.740467 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:26.741209 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.741743 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.742562 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.742776 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.742857 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.744289 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:26.744292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:26.744294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:26.744318 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:26.745659 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.746052 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.746055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.746056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.746137 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:26.746374 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:26.746377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:26.746378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:26.746608 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:26.746715 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.746910 (+ 195us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.747005 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260805 23:19:26.750579 9543 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:26.753780 9543 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:26.768815 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.762816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.23.88.66:37409)
0805 23:19:26.763474 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.763487 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.763513 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:19:26.763754 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.763762 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.763993 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.764159 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:26.764618 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.765633 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.767273 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.768092 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768209 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.768353 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.768539 (+ 186us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.768637 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":121,"threads_started":1}
I20260805 23:19:26.768805 10044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.762675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:37409 (local address 127.0.0.1:39108)
0805 23:19:26.763343 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.763397 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.763532 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.764370 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.764375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.764395 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:26.764496 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.764501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.765760 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.765763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.767145 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.767156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.768212 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.768324 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.768362 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.768509 (+ 147us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.768638 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":34,"thread_start_us":98,"threads_started":1}
I20260805 23:19:26.769340 10045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.762766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39519 (local address 127.0.0.1:58548)
0805 23:19:26.763831 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.763845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.764030 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.764620 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.764622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.764636 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:26.764720 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.764724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.766122 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.766125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.767354 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.767361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.768415 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.768515 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.768538 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.769180 (+ 642us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.769246 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1001,"mutex_wait_us":47,"thread_start_us":94,"threads_started":1}
I20260805 23:19:26.769572 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.763552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:40437 (local address 127.0.0.1:45804)
0805 23:19:26.764724 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.764738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.764837 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.765287 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.765290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.765308 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:26.765404 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.765409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.766660 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.766663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.767896 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.767907 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.769161 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.769165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.769261 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.769301 (+ 40us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.769379 (+ 78us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.769448 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1114,"mutex_wait_us":29,"thread_start_us":107,"threads_started":1}
I20260805 23:19:26.769901 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.763195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58548 (local address 127.23.88.65:39519)
0805 23:19:26.763559 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.763566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.763941 (+ 375us) server_negotiation.cc:407] Connection header received
0805 23:19:26.764194 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.764200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.764382 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.764478 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:26.764834 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.766003 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.767464 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.768092 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768198 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.768349 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.768677 (+ 328us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.768798 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":80,"threads_started":1}
I20260805 23:19:26.769975 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.763797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.23.88.67:40437)
0805 23:19:26.764288 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.764299 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.764870 (+ 571us) server_negotiation.cc:407] Connection header received
0805 23:19:26.764955 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.764959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.765130 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.765279 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:26.765532 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.766504 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768041 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.768871 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.768976 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.769093 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.769441 (+ 348us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.769554 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":189,"threads_started":1}
I20260805 23:19:26.772359 9673 tablet_service.cc:1467] Processing CreateTablet for tablet c65492ad2a654d438bbe67d159aa5c82 (DEFAULT_TABLE table=test-table [id=45bbb75d377045f1b79de87e8ee6a1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:26.772224 9818 tablet_service.cc:1467] Processing CreateTablet for tablet c65492ad2a654d438bbe67d159aa5c82 (DEFAULT_TABLE table=test-table [id=45bbb75d377045f1b79de87e8ee6a1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:26.772653 9961 tablet_service.cc:1467] Processing CreateTablet for tablet c65492ad2a654d438bbe67d159aa5c82 (DEFAULT_TABLE table=test-table [id=45bbb75d377045f1b79de87e8ee6a1bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:26.773344 9673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65492ad2a654d438bbe67d159aa5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:26.773458 9818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65492ad2a654d438bbe67d159aa5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:26.773847 9961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c65492ad2a654d438bbe67d159aa5c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:26.780444 10050 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: Bootstrap starting.
I20260805 23:19:26.781076 10052 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Bootstrap starting.
I20260805 23:19:26.782624 10051 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Bootstrap starting.
I20260805 23:19:26.783183 10052 tablet_bootstrap.cc:654] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:26.783183 10050 tablet_bootstrap.cc:654] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:26.784159 10050 log.cc:824] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:26.784418 10052 log.cc:824] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:26.785140 10051 tablet_bootstrap.cc:654] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:26.785995 10051 log.cc:824] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:26.786576 10050 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: No bootstrap required, opened a new log
I20260805 23:19:26.786576 10052 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: No bootstrap required, opened a new log
I20260805 23:19:26.786778 10050 ts_tablet_manager.cc:1397] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260805 23:19:26.786825 10052 ts_tablet_manager.cc:1397] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260805 23:19:26.787940 10051 tablet_bootstrap.cc:492] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: No bootstrap required, opened a new log
I20260805 23:19:26.788074 10051 ts_tablet_manager.cc:1397] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:19:26.790813 10051 raft_consensus.cc:348] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.790831 10052 raft_consensus.cc:348] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.791136 10052 raft_consensus.cc:374] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:26.791137 10051 raft_consensus.cc:374] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:26.791198 10051 raft_consensus.cc:729] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b22277cf57934680ba40666d79e2bff4, State: Initialized, Role: FOLLOWER
I20260805 23:19:26.791225 10052 raft_consensus.cc:729] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14dea609d9743909d3bdce1669a5db3, State: Initialized, Role: FOLLOWER
I20260805 23:19:26.791247 10050 raft_consensus.cc:348] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.791523 10050 raft_consensus.cc:374] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:26.791594 10050 raft_consensus.cc:729] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e919ce3472a428a8aaa8e01eb1849ca, State: Initialized, Role: FOLLOWER
I20260805 23:19:26.791731 10052 consensus_queue.cc:260] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.791702 10051 consensus_queue.cc:260] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.791954 10050 consensus_queue.cc:260] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.792780 10052 ts_tablet_manager.cc:1428] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:26.792784 10050 ts_tablet_manager.cc:1428] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260805 23:19:26.792781 10051 ts_tablet_manager.cc:1428] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:26.793146 9745 heartbeater.cc:503] Master 127.23.88.126:41123 was elected leader, sending a full tablet report...
I20260805 23:19:26.793378 10031 heartbeater.cc:503] Master 127.23.88.126:41123 was elected leader, sending a full tablet report...
I20260805 23:19:26.794342 9887 heartbeater.cc:503] Master 127.23.88.126:41123 was elected leader, sending a full tablet report...
I20260805 23:19:26.870149 10057 raft_consensus.cc:482] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:26.870358 10057 raft_consensus.cc:504] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.871654 10057 leader_election.cc:288] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e919ce3472a428a8aaa8e01eb1849ca (127.23.88.65:39519), a14dea609d9743909d3bdce1669a5db3 (127.23.88.67:40437)
I20260805 23:19:26.875331 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.872106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:50369 (local address 127.23.88.65:39519)
0805 23:19:26.872235 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.872240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.872256 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:19:26.872322 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.872324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.872364 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.872492 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:26.872783 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.873288 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.874327 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.874750 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.874880 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.874936 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.875127 (+ 191us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.875208 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:19:26.875331 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.871873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39519 (local address 127.23.88.66:50369)
0805 23:19:26.872034 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.872052 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.872162 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.872511 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.872514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.872534 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:26.872656 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.872662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.873406 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.873412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.874188 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.874197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.874875 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.874878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.874995 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.875013 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.875119 (+ 106us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.875189 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 23:19:26.875722 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.872116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46139 (local address 127.23.88.67:40437)
0805 23:19:26.872235 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.872240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.872569 (+ 329us) server_negotiation.cc:407] Connection header received
0805 23:19:26.872719 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.872726 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.872770 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.872851 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:26.873136 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.873637 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.874663 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.875069 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.875241 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.875290 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.875525 (+ 235us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.875603 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:19:26.876080 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.872054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:40437 (local address 127.23.88.66:46139)
0805 23:19:26.872454 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.872466 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:26.872592 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:26.872871 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:26.872874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:26.872890 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:26.873023 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.873028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.873748 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.873754 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.874532 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:26.874540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.875233 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.875236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:26.875337 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.875371 (+ 34us) client_negotiation.cc:769] Sending connection context
0805 23:19:26.875865 (+ 494us) client_negotiation.cc:240] Negotiation successful
0805 23:19:26.875943 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":133,"threads_started":1}
I20260805 23:19:26.876055 9694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65492ad2a654d438bbe67d159aa5c82" candidate_uuid: "b22277cf57934680ba40666d79e2bff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" is_pre_election: true
I20260805 23:19:26.876407 9694 raft_consensus.cc:2454] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22277cf57934680ba40666d79e2bff4 in term 0.
I20260805 23:19:26.876708 9984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65492ad2a654d438bbe67d159aa5c82" candidate_uuid: "b22277cf57934680ba40666d79e2bff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a14dea609d9743909d3bdce1669a5db3" is_pre_election: true
I20260805 23:19:26.876917 9767 leader_election.cc:302] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e919ce3472a428a8aaa8e01eb1849ca, b22277cf57934680ba40666d79e2bff4; no voters:
I20260805 23:19:26.876983 9984 raft_consensus.cc:2454] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b22277cf57934680ba40666d79e2bff4 in term 0.
I20260805 23:19:26.877198 10057 raft_consensus.cc:2781] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:26.877296 10057 raft_consensus.cc:482] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:26.877355 10057 raft_consensus.cc:3037] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:26.878414 10057 raft_consensus.cc:504] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.878806 10057 leader_election.cc:288] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [CANDIDATE]: Term 1 election: Requested vote from peers 4e919ce3472a428a8aaa8e01eb1849ca (127.23.88.65:39519), a14dea609d9743909d3bdce1669a5db3 (127.23.88.67:40437)
I20260805 23:19:26.879237 9694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65492ad2a654d438bbe67d159aa5c82" candidate_uuid: "b22277cf57934680ba40666d79e2bff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e919ce3472a428a8aaa8e01eb1849ca"
I20260805 23:19:26.879374 9694 raft_consensus.cc:3037] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:26.879479 9984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c65492ad2a654d438bbe67d159aa5c82" candidate_uuid: "b22277cf57934680ba40666d79e2bff4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a14dea609d9743909d3bdce1669a5db3"
I20260805 23:19:26.879606 9984 raft_consensus.cc:3037] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:26.880862 9694 raft_consensus.cc:2454] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22277cf57934680ba40666d79e2bff4 in term 1.
I20260805 23:19:26.881179 9767 leader_election.cc:302] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e919ce3472a428a8aaa8e01eb1849ca, b22277cf57934680ba40666d79e2bff4; no voters:
I20260805 23:19:26.881191 9984 raft_consensus.cc:2454] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b22277cf57934680ba40666d79e2bff4 in term 1.
I20260805 23:19:26.881350 10057 raft_consensus.cc:2781] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:26.882025 10057 raft_consensus.cc:686] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 1 LEADER]: Becoming Leader. State: Replica: b22277cf57934680ba40666d79e2bff4, State: Running, Role: LEADER
I20260805 23:19:26.882429 10057 consensus_queue.cc:237] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } }
I20260805 23:19:26.885864 9542 catalog_manager.cc:5495] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b22277cf57934680ba40666d79e2bff4 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "b22277cf57934680ba40666d79e2bff4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b22277cf57934680ba40666d79e2bff4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 37409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 } health_report { overall_health: UNKNOWN } } }
I20260805 23:19:26.922981 10046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.919185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.23.88.66:37409)
0805 23:19:26.919327 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:26.919331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:26.919349 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:19:26.919497 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:26.919499 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:26.919558 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:26.919677 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:26.919941 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.920461 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.921370 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:26.921604 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:26.921723 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:26.922055 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:26.922510 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:26.922652 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:26.922719 (+ 67us) server_negotiation.cc:299] Negotiation successful
0805 23:19:26.922783 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:19:26.929049 9694 raft_consensus.cc:1261] T c65492ad2a654d438bbe67d159aa5c82 P 4e919ce3472a428a8aaa8e01eb1849ca [term 1 FOLLOWER]: Refusing update from remote peer b22277cf57934680ba40666d79e2bff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:26.929049 9984 raft_consensus.cc:1261] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 1 FOLLOWER]: Refusing update from remote peer b22277cf57934680ba40666d79e2bff4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:26.929782 10057 consensus_queue.cc:1035] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a14dea609d9743909d3bdce1669a5db3" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:26.930001 10060 consensus_queue.cc:1035] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e919ce3472a428a8aaa8e01eb1849ca" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:26.939411 10066 mvcc.cc:205] Tried to move back new op lower bound from 7315341176533536768 to 7315341176353144832. Current Snapshot: MvccSnapshot[applied={T|T < 7315341176533536768}]
I20260805 23:19:26.945883 9543 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:26.946098 9543 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:26.948946 9543 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23992a2df7894f8cae1d78257ce1993a: Sending DeleteTablet for 3 replicas of tablet c65492ad2a654d438bbe67d159aa5c82
I20260805 23:19:26.949901 9818 tablet_service.cc:1514] Processing DeleteTablet for tablet c65492ad2a654d438bbe67d159aa5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:39108
I20260805 23:19:26.949930 9673 tablet_service.cc:1514] Processing DeleteTablet for tablet c65492ad2a654d438bbe67d159aa5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:58548
I20260805 23:19:26.950102 9961 tablet_service.cc:1514] Processing DeleteTablet for tablet c65492ad2a654d438bbe67d159aa5c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:26 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:45804
I20260805 23:19:26.950493 10075 tablet_replica.cc:330] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: stopping tablet replica
I20260805 23:19:26.950675 10077 tablet_replica.cc:330] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: stopping tablet replica
I20260805 23:19:26.950959 10075 raft_consensus.cc:2229] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:19:26.951057 10077 raft_consensus.cc:2229] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:26.951426 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9607
I20260805 23:19:26.951514 10077 raft_consensus.cc:2258] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:26.951479 10075 raft_consensus.cc:2258] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:26.954234 10077 ts_tablet_manager.cc:1905] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:26.956540 10075 ts_tablet_manager.cc:1905] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:26.959295 10075 ts_tablet_manager.cc:1918] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:26.961580 10077 ts_tablet_manager.cc:1918] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:26.961782 10075 log.cc:1196] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/c65492ad2a654d438bbe67d159aa5c82
I20260805 23:19:26.962219 10075 ts_tablet_manager.cc:1939] T c65492ad2a654d438bbe67d159aa5c82 P b22277cf57934680ba40666d79e2bff4: Deleting consensus metadata
I20260805 23:19:26.962376 10077 log.cc:1196] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/c65492ad2a654d438bbe67d159aa5c82
I20260805 23:19:26.962774 10077 ts_tablet_manager.cc:1939] T c65492ad2a654d438bbe67d159aa5c82 P a14dea609d9743909d3bdce1669a5db3: Deleting consensus metadata
I20260805 23:19:26.964224 9528 catalog_manager.cc:4841] TS b22277cf57934680ba40666d79e2bff4 (127.23.88.66:37409): tablet c65492ad2a654d438bbe67d159aa5c82 (table test-table [id=45bbb75d377045f1b79de87e8ee6a1bd]) successfully deleted
I20260805 23:19:26.965814 9526 catalog_manager.cc:4841] TS a14dea609d9743909d3bdce1669a5db3 (127.23.88.67:40437): tablet c65492ad2a654d438bbe67d159aa5c82 (table test-table [id=45bbb75d377045f1b79de87e8ee6a1bd]) successfully deleted
W20260805 23:19:26.966365 9527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:39519 (error 108)
I20260805 23:19:26.967483 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9750
I20260805 23:19:26.969441 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:26.967861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:39519 (local address 127.0.0.1:58550)
0805 23:19:26.969242 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:26.969327 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:39519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1272}
W20260805 23:19:26.970582 9527 catalog_manager.cc:4584] TS 4e919ce3472a428a8aaa8e01eb1849ca (127.23.88.65:39519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c65492ad2a654d438bbe67d159aa5c82: Network error: Client connection negotiation failed: client connection to 127.23.88.65:39519: connect: Connection refused (error 111)
I20260805 23:19:26.976079 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9893
I20260805 23:19:26.983283 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 9509
2026-08-05T23:19:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2446 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-05T23:19:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:27Z Disabled control of system clock
I20260805 23:19:27.016355 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44543
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:44543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:27.132071 10082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:27.132361 10082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:27.132400 10082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:27.136590 10082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:27.136665 10082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:27.136680 10082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:27.136698 10082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:27.136713 10082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:27.142212 10082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:44543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:27.143671 10082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:27.145133 10082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:27.153839 10087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.153865 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.153972 10082 server_base.cc:1034] running on GCE node
W20260805 23:19:27.153803 10090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.154722 10082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:27.155877 10082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:27.157095 10082 hybrid_clock.cc:648] HybridClock initialized: now 1785971967157074 us; error 45 us; skew 500 ppm
I20260805 23:19:27.159058 10082 webserver.cc:460] Webserver started at http://127.23.88.126:38269/ using document root <none> and password file <none>
I20260805 23:19:27.159706 10082 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:27.159798 10082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:27.160044 10082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:27.162146 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "e8c533a9adb8409a800cafb7f1ce71af"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "f68aa5f385cfcb5d2556eadd5ef97b62"
server_key_iv: "d3a8a11d0fd880bf17da877d649c59bc"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.162725 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "e8c533a9adb8409a800cafb7f1ce71af"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "f68aa5f385cfcb5d2556eadd5ef97b62"
server_key_iv: "d3a8a11d0fd880bf17da877d649c59bc"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.166531 10082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:19:27.169086 10096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.170282 10082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:27.170428 10082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "e8c533a9adb8409a800cafb7f1ce71af"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "f68aa5f385cfcb5d2556eadd5ef97b62"
server_key_iv: "d3a8a11d0fd880bf17da877d649c59bc"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.170548 10082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:27.182826 10082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:27.183640 10082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:27.183856 10082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:27.191407 10082 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:44543
I20260805 23:19:27.191437 10149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:44543 every 8 connection(s)
I20260805 23:19:27.192442 10082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:27.192811 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10082
I20260805 23:19:27.192962 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:27.193291 23905 external_mini_cluster.cc:1428] Setting key dca08fd9afe5e1770f7cc0f774d35148
I20260805 23:19:27.196130 10150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:27.201922 10150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: Bootstrap starting.
I20260805 23:19:27.202893 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.194834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55156 (local address 127.23.88.126:44543)
0805 23:19:27.195523 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.195547 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.195583 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:19:27.196273 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.196295 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.196702 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.197042 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:27.197474 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.198455 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.199464 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.199974 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.200147 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.201074 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:27.201079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:27.201092 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:27.201494 (+ 402us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:27.201554 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:27.201573 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:27.201682 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.201809 (+ 127us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.202142 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":181,"threads_started":1}
I20260805 23:19:27.204663 10150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:27.205617 10150 log.cc:824] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:27.207830 10150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: No bootstrap required, opened a new log
I20260805 23:19:27.211939 10150 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } }
I20260805 23:19:27.212177 10150 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:27.212211 10150 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8c533a9adb8409a800cafb7f1ce71af, State: Initialized, Role: FOLLOWER
I20260805 23:19:27.212623 10150 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } }
I20260805 23:19:27.212747 10150 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:27.212790 10150 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:27.212864 10150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:27.213766 10150 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } }
I20260805 23:19:27.214064 10150 leader_election.cc:302] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8c533a9adb8409a800cafb7f1ce71af; no voters:
I20260805 23:19:27.214331 10150 leader_election.cc:288] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:27.214531 10156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:27.214785 10156 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [term 1 LEADER]: Becoming Leader. State: Replica: e8c533a9adb8409a800cafb7f1ce71af, State: Running, Role: LEADER
I20260805 23:19:27.215131 10156 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } }
I20260805 23:19:27.215734 10150 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:27.216967 10157 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8c533a9adb8409a800cafb7f1ce71af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } } }
I20260805 23:19:27.217144 10157 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:27.216967 10158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8c533a9adb8409a800cafb7f1ce71af. Latest consensus state: current_term: 1 leader_uuid: "e8c533a9adb8409a800cafb7f1ce71af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8c533a9adb8409a800cafb7f1ce71af" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44543 } } }
I20260805 23:19:27.217401 10158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af [sys.catalog]: This master's current role is: LEADER
W20260805 23:19:27.220764 10171 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:19:27.220863 10171 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:19:27.220942 10165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:27.221661 10165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:27.227725 10165 catalog_manager.cc:1347] Generated new cluster ID: 46b27e27696947a19d2e0f1a7b6d07e7
I20260805 23:19:27.227819 10165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:27.239010 10165 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:27.240322 10165 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:27.247201 10165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: Generated new TSK 0
I20260805 23:19:27.248072 10165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:27.253507 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--builtin_ntp_servers=127.23.88.84:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:27.369619 10175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:27.369926 10175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:27.370002 10175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:27.374256 10175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:27.374341 10175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:27.374449 10175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:27.379604 10175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:27.380810 10175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:27.382287 10175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:27.392555 10181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.392630 10175 server_base.cc:1034] running on GCE node
W20260805 23:19:27.392554 10183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.392551 10180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.393246 10175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:27.393985 10175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:27.395162 10175 hybrid_clock.cc:648] HybridClock initialized: now 1785971967395122 us; error 46 us; skew 500 ppm
I20260805 23:19:27.396997 10175 webserver.cc:460] Webserver started at http://127.23.88.65:40137/ using document root <none> and password file <none>
I20260805 23:19:27.397604 10175 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:27.397702 10175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:27.397948 10175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:27.400166 10175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "d432351c561a426993a00491a19548e1"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "48b116b563c7d5d2fcb12f5e15bf068e"
server_key_iv: "5e4a0e0c643a39f059bcd788cd7c8b97"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.400766 10175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "d432351c561a426993a00491a19548e1"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "48b116b563c7d5d2fcb12f5e15bf068e"
server_key_iv: "5e4a0e0c643a39f059bcd788cd7c8b97"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.404773 10175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:27.407616 10189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.408994 10175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:27.409152 10175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "d432351c561a426993a00491a19548e1"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "48b116b563c7d5d2fcb12f5e15bf068e"
server_key_iv: "5e4a0e0c643a39f059bcd788cd7c8b97"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.409278 10175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:27.445281 10175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:27.446049 10175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:27.446271 10175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:27.446914 10175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:27.447964 10175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:27.448035 10175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.448101 10175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:27.448153 10175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.457751 10175 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:43243
I20260805 23:19:27.457782 10310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:43243 every 8 connection(s)
I20260805 23:19:27.458691 10175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:27.460858 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10175
I20260805 23:19:27.461031 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:27.461346 23905 external_mini_cluster.cc:1428] Setting key 629b3c9f49edfff8d69b05743f952ca4
I20260805 23:19:27.464641 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--builtin_ntp_servers=127.23.88.84:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:27.468840 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.460708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:46315 (local address 127.23.88.126:44543)
0805 23:19:27.460885 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.460889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.462224 (+ 1335us) server_negotiation.cc:407] Connection header received
0805 23:19:27.463860 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.463863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.463914 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.463994 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:27.464992 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.465545 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.466756 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.466989 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.467123 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.468312 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:27.468316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:27.468318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:27.468332 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:27.468359 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:27.468367 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:27.468447 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.468606 (+ 159us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.468691 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260805 23:19:27.469841 10313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.461247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44543 (local address 127.23.88.65:46315)
0805 23:19:27.462043 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.462099 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.463603 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.464169 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.464175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.464262 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:27.464837 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.464858 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.465698 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.465707 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.466624 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.466634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.467172 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.467178 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.467350 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.468034 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:27.468035 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:27.468076 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:27.468142 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:27.468450 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:27.468463 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:27.468469 (+ 6us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.468606 (+ 137us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.468903 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":169,"threads_started":1}
I20260805 23:19:27.471158 10311 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44543
I20260805 23:19:27.471442 10311 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:27.472162 10311 heartbeater.cc:511] Master 127.23.88.126:44543 requested a full tablet report, sending...
I20260805 23:19:27.473969 10113 ts_manager.cc:194] Registered new tserver with Master: d432351c561a426993a00491a19548e1 (127.23.88.65:43243)
I20260805 23:19:27.475288 10113 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.65:46315
W20260805 23:19:27.581027 10314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:27.581307 10314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:27.581347 10314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:27.585572 10314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:27.585640 10314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:27.585698 10314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:27.593767 10314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:27.595449 10314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:27.597261 10314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:27.607816 10320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.607816 10322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.607810 10319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.608105 10314 server_base.cc:1034] running on GCE node
I20260805 23:19:27.608624 10314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:27.609323 10314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:27.610535 10314 hybrid_clock.cc:648] HybridClock initialized: now 1785971967610522 us; error 31 us; skew 500 ppm
I20260805 23:19:27.612792 10314 webserver.cc:460] Webserver started at http://127.23.88.66:33393/ using document root <none> and password file <none>
I20260805 23:19:27.613561 10314 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:27.613653 10314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:27.613885 10314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:27.616243 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "ddf979cdd9ed4647afe0b69a4ac297c4"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "6ed05021ab6c84834d7de3474eb4e5ad"
server_key_iv: "1d3475b3a8fd2ac88efddb5fec35e41c"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.616951 10314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "ddf979cdd9ed4647afe0b69a4ac297c4"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "6ed05021ab6c84834d7de3474eb4e5ad"
server_key_iv: "1d3475b3a8fd2ac88efddb5fec35e41c"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.621028 10314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:19:27.623665 10328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.624977 10314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:27.625087 10314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "ddf979cdd9ed4647afe0b69a4ac297c4"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "6ed05021ab6c84834d7de3474eb4e5ad"
server_key_iv: "1d3475b3a8fd2ac88efddb5fec35e41c"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.625240 10314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:27.653736 10314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:27.654536 10314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:27.654757 10314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:27.655463 10314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:27.656507 10314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:27.656564 10314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.656637 10314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:27.656677 10314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.666817 10314 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:38387
I20260805 23:19:27.666843 10448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:38387 every 8 connection(s)
I20260805 23:19:27.667891 10314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:27.671892 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10314
I20260805 23:19:27.672005 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:27.672322 23905 external_mini_cluster.cc:1428] Setting key 44fa7a0b8146aea96757c96d649ecf87
I20260805 23:19:27.674772 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--builtin_ntp_servers=127.23.88.84:39497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:27.678375 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.669954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:43835 (local address 127.23.88.126:44543)
0805 23:19:27.670118 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.670123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.671293 (+ 1170us) server_negotiation.cc:407] Connection header received
0805 23:19:27.672815 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.672821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.672893 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.672970 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:27.674176 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.675042 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.676240 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.676484 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.676598 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.677816 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:27.677819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:27.677821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:27.677834 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:27.677855 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:27.677863 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:27.677933 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.678147 (+ 214us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.678215 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:19:27.679589 10451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.670424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44543 (local address 127.23.88.66:43835)
0805 23:19:27.671127 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.671189 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.672603 (+ 1414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.673175 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.673181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.673265 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:27.673908 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.673929 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.675232 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.675235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.676109 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.676118 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.676642 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.676646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.676812 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.677501 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:27.677505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:27.677561 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:27.677645 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:27.677960 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:27.677972 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:27.677981 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.678111 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.678417 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":115,"threads_started":1}
I20260805 23:19:27.680867 10449 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44543
I20260805 23:19:27.681157 10449 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:27.681954 10449 heartbeater.cc:511] Master 127.23.88.126:44543 requested a full tablet report, sending...
I20260805 23:19:27.683269 10113 ts_manager.cc:194] Registered new tserver with Master: ddf979cdd9ed4647afe0b69a4ac297c4 (127.23.88.66:38387)
I20260805 23:19:27.683915 10113 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.66:43835
W20260805 23:19:27.797066 10452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:27.797416 10452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:27.797508 10452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:27.801821 10452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:27.801927 10452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:27.802030 10452 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:27.807413 10452 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:27.808666 10452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:27.810137 10452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:19:27.821163 10452 server_base.cc:1034] running on GCE node
W20260805 23:19:27.821123 10460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.821118 10457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:27.821118 10458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:27.821935 10452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:27.822852 10452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:27.824199 10452 hybrid_clock.cc:648] HybridClock initialized: now 1785971967824185 us; error 59 us; skew 500 ppm
I20260805 23:19:27.826436 10452 webserver.cc:460] Webserver started at http://127.23.88.67:34615/ using document root <none> and password file <none>
I20260805 23:19:27.827150 10452 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:27.827272 10452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:27.827526 10452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:27.829749 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "31e3efe131cc4387b02ad18440c727c2"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "5c148f3ec8bb7251ad0b92991cb05b39"
server_key_iv: "f3db5925e5bc35c9c144dad4218786a0"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.830399 10452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "31e3efe131cc4387b02ad18440c727c2"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "5c148f3ec8bb7251ad0b92991cb05b39"
server_key_iv: "f3db5925e5bc35c9c144dad4218786a0"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.834558 10452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:19:27.837584 10466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.838973 10452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:19:27.839177 10452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "31e3efe131cc4387b02ad18440c727c2"
format_stamp: "Formatted at 2026-08-05 23:19:27 on dist-test-slave-vkjp"
server_key: "5c148f3ec8bb7251ad0b92991cb05b39"
server_key_iv: "f3db5925e5bc35c9c144dad4218786a0"
server_key_version: "encryptionkey@0"
I20260805 23:19:27.839324 10452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:27.858512 10452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:27.859407 10452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:27.859622 10452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:27.860255 10452 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:27.861284 10452 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:27.861362 10452 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.861449 10452 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:27.861498 10452 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:27.870764 10452 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:33859
I20260805 23:19:27.870787 10589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:33859 every 8 connection(s)
I20260805 23:19:27.871865 10452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:27.872946 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10452
I20260805 23:19:27.873064 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:27.873361 23905 external_mini_cluster.cc:1428] Setting key 763ea514e291587b8721b8b3369a7113
I20260805 23:19:27.882573 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.874251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:45889 (local address 127.23.88.126:44543)
0805 23:19:27.874409 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.874414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.876003 (+ 1589us) server_negotiation.cc:407] Connection header received
0805 23:19:27.877495 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.877501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.877565 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.877685 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:27.878623 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.879282 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.880536 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.880755 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.880838 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.881987 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:27.881990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:27.881992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:27.882006 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:27.882025 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:27.882033 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:27.882093 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.882323 (+ 230us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.882410 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:19:27.883890 10592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.874867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44543 (local address 127.23.88.67:45889)
0805 23:19:27.875819 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.875906 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.877237 (+ 1331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.877826 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.877833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.877926 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:27.878471 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.878486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.879403 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.879405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.880387 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.880398 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.880903 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.880906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.881074 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.881711 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:27.881713 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:27.881757 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:27.881830 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:27.882110 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:27.882117 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:27.882121 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.882284 (+ 163us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.882606 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":225,"threads_started":1}
I20260805 23:19:27.885476 10590 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44543
I20260805 23:19:27.885776 10590 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:27.886397 10590 heartbeater.cc:511] Master 127.23.88.126:44543 requested a full tablet report, sending...
I20260805 23:19:27.887631 10112 ts_manager.cc:194] Registered new tserver with Master: 31e3efe131cc4387b02ad18440c727c2 (127.23.88.67:33859)
I20260805 23:19:27.888263 10112 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.67:45889
I20260805 23:19:27.900139 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:27.906728 10154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.902424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55160 (local address 127.23.88.126:44543)
0805 23:19:27.902588 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.902592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.902659 (+ 67us) server_negotiation.cc:407] Connection header received
0805 23:19:27.902787 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.902789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.902833 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.902901 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:27.903395 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.903938 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.904948 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.905172 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.905289 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.906245 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:27.906248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:27.906250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:27.906263 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:27.906285 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:27.906293 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:27.906375 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.906497 (+ 122us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.906558 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:19:27.909988 10112 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:27.913223 10112 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:27.928634 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.922520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44534 (local address 127.23.88.67:33859)
0805 23:19:27.923708 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.923722 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.923748 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:19:27.923826 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.923829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.924063 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.924260 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:27.924558 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925751 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.927376 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.927941 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.928069 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.928271 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.928354 (+ 83us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.928468 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1079,"thread_start_us":89,"threads_started":1}
I20260805 23:19:27.928792 10602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.922503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:43243 (local address 127.0.0.1:51686)
0805 23:19:27.923302 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.923319 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.923428 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.924395 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.924398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.924414 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:27.924529 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.924535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925654 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.925661 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.927263 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.927271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.928306 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.928310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.928421 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.928445 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.928495 (+ 50us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.928575 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":135,"thread_start_us":109,"threads_started":1}
I20260805 23:19:27.929023 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.923188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.23.88.66:38387)
0805 23:19:27.923624 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.923637 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.923966 (+ 329us) server_negotiation.cc:407] Connection header received
0805 23:19:27.924056 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.924061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.924236 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.924384 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:27.924758 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925868 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.927224 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.928051 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.928160 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.928360 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.928729 (+ 369us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.928863 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":112,"threads_started":1}
I20260805 23:19:27.928653 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.922321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:33859 (local address 127.0.0.1:44534)
0805 23:19:27.923187 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.923244 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.923385 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.924249 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.924256 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.924278 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:27.924440 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.924447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925875 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.925878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.927222 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.927233 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.928066 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.928069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.928176 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.928214 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.928324 (+ 110us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.928453 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":111,"thread_start_us":95,"threads_started":1}
I20260805 23:19:27.929296 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.922735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38387 (local address 127.0.0.1:47700)
0805 23:19:27.923632 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:27.923645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:27.923756 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:27.924530 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:27.924532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:27.924546 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:27.924643 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.924649 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925983 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.925989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.927064 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:27.927075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.928416 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.928419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:27.928503 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.928521 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:19:27.928580 (+ 59us) client_negotiation.cc:240] Negotiation successful
0805 23:19:27.928632 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":4,"thread_start_us":102,"threads_started":1}
I20260805 23:19:27.929824 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:27.923035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51686 (local address 127.23.88.65:43243)
0805 23:19:27.923986 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:27.923993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:27.924014 (+ 21us) server_negotiation.cc:407] Connection header received
0805 23:19:27.924080 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:27.924084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:27.924236 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:27.924678 (+ 442us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:27.924826 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.925543 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.927402 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:27.928172 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:27.929250 (+ 1078us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:27.929510 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:27.929563 (+ 53us) server_negotiation.cc:299] Negotiation successful
0805 23:19:27.929670 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":137,"threads_started":1}
I20260805 23:19:27.932132 10516 tablet_service.cc:1467] Processing CreateTablet for tablet af94c63f90ab48c9a529486d75203a18 (DEFAULT_TABLE table=test-table [id=685e338f92a347e3bbf993d7e96d7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:27.932260 10242 tablet_service.cc:1467] Processing CreateTablet for tablet af94c63f90ab48c9a529486d75203a18 (DEFAULT_TABLE table=test-table [id=685e338f92a347e3bbf993d7e96d7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:27.932261 10383 tablet_service.cc:1467] Processing CreateTablet for tablet af94c63f90ab48c9a529486d75203a18 (DEFAULT_TABLE table=test-table [id=685e338f92a347e3bbf993d7e96d7514]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:27.933193 10516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94c63f90ab48c9a529486d75203a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:27.933300 10383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94c63f90ab48c9a529486d75203a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:27.933300 10242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af94c63f90ab48c9a529486d75203a18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:27.940114 10608 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Bootstrap starting.
I20260805 23:19:27.941526 10610 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Bootstrap starting.
I20260805 23:19:27.942529 10608 tablet_bootstrap.cc:654] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:27.943506 10610 tablet_bootstrap.cc:654] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:27.943514 10608 log.cc:824] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:27.943590 10609 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Bootstrap starting.
I20260805 23:19:27.944332 10610 log.cc:824] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:27.945809 10609 tablet_bootstrap.cc:654] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:27.945904 10608 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: No bootstrap required, opened a new log
I20260805 23:19:27.946126 10608 ts_tablet_manager.cc:1397] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:19:27.946686 10609 log.cc:824] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:27.947189 10610 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: No bootstrap required, opened a new log
I20260805 23:19:27.947394 10610 ts_tablet_manager.cc:1397] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:27.948819 10609 tablet_bootstrap.cc:492] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: No bootstrap required, opened a new log
I20260805 23:19:27.949048 10609 ts_tablet_manager.cc:1397] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:27.950440 10610 raft_consensus.cc:348] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.950698 10610 raft_consensus.cc:374] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:27.950752 10610 raft_consensus.cc:729] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf979cdd9ed4647afe0b69a4ac297c4, State: Initialized, Role: FOLLOWER
I20260805 23:19:27.950767 10608 raft_consensus.cc:348] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.951030 10608 raft_consensus.cc:374] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:27.951267 10610 consensus_queue.cc:260] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.951390 10608 raft_consensus.cc:729] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31e3efe131cc4387b02ad18440c727c2, State: Initialized, Role: FOLLOWER
I20260805 23:19:27.951828 10608 consensus_queue.cc:260] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.952382 10610 ts_tablet_manager.cc:1428] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:19:27.952749 10449 heartbeater.cc:503] Master 127.23.88.126:44543 was elected leader, sending a full tablet report...
I20260805 23:19:27.952822 10608 ts_tablet_manager.cc:1428] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260805 23:19:27.953173 10590 heartbeater.cc:503] Master 127.23.88.126:44543 was elected leader, sending a full tablet report...
I20260805 23:19:27.954367 10609 raft_consensus.cc:348] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.954655 10609 raft_consensus.cc:374] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:27.954718 10609 raft_consensus.cc:729] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d432351c561a426993a00491a19548e1, State: Initialized, Role: FOLLOWER
I20260805 23:19:27.955174 10609 consensus_queue.cc:260] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:27.956046 10609 ts_tablet_manager.cc:1428] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:19:27.956465 10311 heartbeater.cc:503] Master 127.23.88.126:44543 was elected leader, sending a full tablet report...
W20260805 23:19:27.960011 10312 tablet.cc:2369] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:28.021715 10617 raft_consensus.cc:482] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:28.021968 10617 raft_consensus.cc:504] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:28.023504 10617 leader_election.cc:288] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31e3efe131cc4387b02ad18440c727c2 (127.23.88.67:33859), ddf979cdd9ed4647afe0b69a4ac297c4 (127.23.88.66:38387)
I20260805 23:19:28.029131 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.023822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:37553 (local address 127.23.88.67:33859)
0805 23:19:28.023969 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:28.023976 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:28.024190 (+ 214us) server_negotiation.cc:407] Connection header received
0805 23:19:28.024361 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:28.024364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:28.024417 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:28.024511 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:28.025021 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.025812 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.027764 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.028374 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.028520 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:28.028574 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:28.028908 (+ 334us) server_negotiation.cc:299] Negotiation successful
0805 23:19:28.029020 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:19:28.029131 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.023931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:40405 (local address 127.23.88.66:38387)
0805 23:19:28.024126 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:28.024130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:28.024169 (+ 39us) server_negotiation.cc:407] Connection header received
0805 23:19:28.024361 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:28.024364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:28.024417 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:28.024531 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:28.025021 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.025825 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.027764 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.028374 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.028540 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:28.028581 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:28.028908 (+ 327us) server_negotiation.cc:299] Negotiation successful
0805 23:19:28.028988 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260805 23:19:28.029212 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.023701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:33859 (local address 127.23.88.65:37553)
0805 23:19:28.024075 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:28.024092 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:28.024227 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:28.024585 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:28.024589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:28.024619 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:28.024884 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:28.024889 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.025945 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.025949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:28.027614 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:28.027627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.028516 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.028519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:28.028727 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:28.028791 (+ 64us) client_negotiation.cc:769] Sending connection context
0805 23:19:28.028878 (+ 87us) client_negotiation.cc:240] Negotiation successful
0805 23:19:28.029021 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":269,"spinlock_wait_cycles":31360,"thread_start_us":113,"threads_started":1}
I20260805 23:19:28.029640 10620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.023808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38387 (local address 127.23.88.65:40405)
0805 23:19:28.024085 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:28.024097 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:28.024227 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:28.024585 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:28.024589 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:28.024619 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:28.024877 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:28.024886 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.025945 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.025949 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:28.027615 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:28.027627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.028544 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.028548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:28.028732 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:28.028788 (+ 56us) client_negotiation.cc:769] Sending connection context
0805 23:19:28.028876 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:19:28.029021 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":324,"spinlock_wait_cycles":19328,"thread_start_us":128,"threads_started":1}
I20260805 23:19:28.029968 10541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94c63f90ab48c9a529486d75203a18" candidate_uuid: "d432351c561a426993a00491a19548e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e3efe131cc4387b02ad18440c727c2" is_pre_election: true
I20260805 23:19:28.030292 10541 raft_consensus.cc:2454] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d432351c561a426993a00491a19548e1 in term 0.
I20260805 23:19:28.030301 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94c63f90ab48c9a529486d75203a18" candidate_uuid: "d432351c561a426993a00491a19548e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" is_pre_election: true
I20260805 23:19:28.030582 10403 raft_consensus.cc:2454] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d432351c561a426993a00491a19548e1 in term 0.
I20260805 23:19:28.030833 10194 leader_election.cc:302] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e3efe131cc4387b02ad18440c727c2, d432351c561a426993a00491a19548e1; no voters:
I20260805 23:19:28.031153 10617 raft_consensus.cc:2781] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:28.031247 10617 raft_consensus.cc:482] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:28.031293 10617 raft_consensus.cc:3037] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:28.032383 10617 raft_consensus.cc:504] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:28.032759 10617 leader_election.cc:288] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [CANDIDATE]: Term 1 election: Requested vote from peers 31e3efe131cc4387b02ad18440c727c2 (127.23.88.67:33859), ddf979cdd9ed4647afe0b69a4ac297c4 (127.23.88.66:38387)
I20260805 23:19:28.033133 10541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94c63f90ab48c9a529486d75203a18" candidate_uuid: "d432351c561a426993a00491a19548e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31e3efe131cc4387b02ad18440c727c2"
I20260805 23:19:28.033253 10403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af94c63f90ab48c9a529486d75203a18" candidate_uuid: "d432351c561a426993a00491a19548e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4"
I20260805 23:19:28.033272 10541 raft_consensus.cc:3037] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:28.033421 10403 raft_consensus.cc:3037] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:28.034386 10541 raft_consensus.cc:2454] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d432351c561a426993a00491a19548e1 in term 1.
I20260805 23:19:28.034578 10403 raft_consensus.cc:2454] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d432351c561a426993a00491a19548e1 in term 1.
I20260805 23:19:28.034759 10194 leader_election.cc:302] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31e3efe131cc4387b02ad18440c727c2, d432351c561a426993a00491a19548e1; no voters:
I20260805 23:19:28.035115 10617 raft_consensus.cc:2781] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:28.035346 10617 raft_consensus.cc:686] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [term 1 LEADER]: Becoming Leader. State: Replica: d432351c561a426993a00491a19548e1, State: Running, Role: LEADER
I20260805 23:19:28.035717 10617 consensus_queue.cc:237] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } }
I20260805 23:19:28.039332 10111 catalog_manager.cc:5495] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d432351c561a426993a00491a19548e1 (127.23.88.65). New cstate: current_term: 1 leader_uuid: "d432351c561a426993a00491a19548e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d432351c561a426993a00491a19548e1" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 43243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 } health_report { overall_health: UNKNOWN } } }
I20260805 23:19:28.053673 10605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.049905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51688 (local address 127.23.88.65:43243)
0805 23:19:28.050055 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:28.050060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:28.050076 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:19:28.050227 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:28.050230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:28.050289 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:28.050402 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:28.050650 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.051234 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.052133 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:28.052360 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:28.052474 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:28.052803 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:28.053245 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:28.053384 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:28.053456 (+ 72us) server_negotiation.cc:299] Negotiation successful
0805 23:19:28.053518 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:19:28.060153 10403 raft_consensus.cc:1261] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 1 FOLLOWER]: Refusing update from remote peer d432351c561a426993a00491a19548e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:28.060153 10541 raft_consensus.cc:1261] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 1 FOLLOWER]: Refusing update from remote peer d432351c561a426993a00491a19548e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:28.060936 10621 consensus_queue.cc:1035] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31e3efe131cc4387b02ad18440c727c2" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:28.061158 10617 consensus_queue.cc:1035] T af94c63f90ab48c9a529486d75203a18 P d432351c561a426993a00491a19548e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf979cdd9ed4647afe0b69a4ac297c4" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:28.070235 10625 mvcc.cc:205] Tried to move back new op lower bound from 7315341181166252032 to 7315341181076987904. Current Snapshot: MvccSnapshot[applied={T|T < 7315341181166252032}]
I20260805 23:19:28.070804 10626 mvcc.cc:205] Tried to move back new op lower bound from 7315341181166252032 to 7315341181076987904. Current Snapshot: MvccSnapshot[applied={T|T < 7315341181166252032}]
I20260805 23:19:28.078918 10112 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:28.079260 10112 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:28.082074 10112 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8c533a9adb8409a800cafb7f1ce71af: Sending DeleteTablet for 3 replicas of tablet af94c63f90ab48c9a529486d75203a18
I20260805 23:19:28.083070 10242 tablet_service.cc:1514] Processing DeleteTablet for tablet af94c63f90ab48c9a529486d75203a18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:28 UTC) from {username='slave'} at 127.0.0.1:51686
I20260805 23:19:28.083227 10516 tablet_service.cc:1514] Processing DeleteTablet for tablet af94c63f90ab48c9a529486d75203a18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:28 UTC) from {username='slave'} at 127.0.0.1:44534
I20260805 23:19:28.083263 10383 tablet_service.cc:1514] Processing DeleteTablet for tablet af94c63f90ab48c9a529486d75203a18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:28 UTC) from {username='slave'} at 127.0.0.1:47700
I20260805 23:19:28.083859 10634 tablet_replica.cc:330] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: stopping tablet replica
I20260805 23:19:28.083966 10636 tablet_replica.cc:330] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: stopping tablet replica
I20260805 23:19:28.084218 10636 raft_consensus.cc:2229] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:28.084218 10634 raft_consensus.cc:2229] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:28.084568 10634 raft_consensus.cc:2258] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:28.084568 10636 raft_consensus.cc:2258] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:28.084692 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10175
I20260805 23:19:28.086459 10636 ts_tablet_manager.cc:1905] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:28.088724 10634 ts_tablet_manager.cc:1905] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:28.089599 10636 ts_tablet_manager.cc:1918] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:28.089713 10636 log.cc:1196] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/af94c63f90ab48c9a529486d75203a18
I20260805 23:19:28.090061 10636 ts_tablet_manager.cc:1939] T af94c63f90ab48c9a529486d75203a18 P ddf979cdd9ed4647afe0b69a4ac297c4: Deleting consensus metadata
I20260805 23:19:28.091529 10634 ts_tablet_manager.cc:1918] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:28.091634 10634 log.cc:1196] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/af94c63f90ab48c9a529486d75203a18
I20260805 23:19:28.091627 10099 catalog_manager.cc:4841] TS ddf979cdd9ed4647afe0b69a4ac297c4 (127.23.88.66:38387): tablet af94c63f90ab48c9a529486d75203a18 (table test-table [id=685e338f92a347e3bbf993d7e96d7514]) successfully deleted
I20260805 23:19:28.091953 10634 ts_tablet_manager.cc:1939] T af94c63f90ab48c9a529486d75203a18 P 31e3efe131cc4387b02ad18440c727c2: Deleting consensus metadata
W20260805 23:19:28.093024 10098 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:43243 (error 108)
I20260805 23:19:28.093041 10100 catalog_manager.cc:4841] TS 31e3efe131cc4387b02ad18440c727c2 (127.23.88.67:33859): tablet af94c63f90ab48c9a529486d75203a18 (table test-table [id=685e338f92a347e3bbf993d7e96d7514]) successfully deleted
I20260805 23:19:28.093950 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10314
I20260805 23:19:28.094245 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:28.093849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:43243 (local address 127.0.0.1:51700)
0805 23:19:28.094002 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:28.094118 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:43243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260805 23:19:28.094985 10098 catalog_manager.cc:4584] TS d432351c561a426993a00491a19548e1 (127.23.88.65:43243): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af94c63f90ab48c9a529486d75203a18: Network error: Client connection negotiation failed: client connection to 127.23.88.65:43243: connect: Connection refused (error 111)
I20260805 23:19:28.103873 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10452
I20260805 23:19:28.110226 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10082
2026-08-05T23:19:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1126 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:28 dist-test-slave-vkjp krb5kdc[10640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:28 dist-test-slave-vkjp krb5kdc[10640](info): set up 2 sockets
Aug 05 23:19:28 dist-test-slave-vkjp krb5kdc[10640](info): commencing operation
krb5kdc: starting...
W20260805 23:19:29.182020 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:29.318397 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44937
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:44937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:29.434580 10655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:29.434918 10655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:29.435012 10655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:29.439198 10655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:29.439309 10655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:29.439349 10655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:29.439388 10655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:29.439419 10655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:29.445135 10655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:44937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:29.446326 10655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:29.447454 10655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:29.456199 10663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:29.456229 10661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.456437 10655 server_base.cc:1034] running on GCE node
W20260805 23:19:29.456226 10660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.457190 10655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:29.458405 10655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:29.459581 10655 hybrid_clock.cc:648] HybridClock initialized: now 1785971969459547 us; error 50 us; skew 500 ppm
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:29.462636 10655 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:29.463657 10655 webserver.cc:460] Webserver started at http://127.23.88.126:43025/ using document root <none> and password file <none>
I20260805 23:19:29.464242 10655 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:29.464304 10655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:29.464535 10655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:29.466728 10655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "798b513f63694673bfcf40fa77824a58"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "6675aa71c5eb5ea8164bfb4056146df5"
server_key_iv: "a3da5d5b5ed626004ada9eb601843da3"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.467360 10655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "798b513f63694673bfcf40fa77824a58"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "6675aa71c5eb5ea8164bfb4056146df5"
server_key_iv: "a3da5d5b5ed626004ada9eb601843da3"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.471405 10655 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:29.473929 10670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:29.475252 10655 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:29.475410 10655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "798b513f63694673bfcf40fa77824a58"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "6675aa71c5eb5ea8164bfb4056146df5"
server_key_iv: "a3da5d5b5ed626004ada9eb601843da3"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.475551 10655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:29.490733 10655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:29.494274 10655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:29.494503 10655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:29.501957 10655 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:44937
I20260805 23:19:29.501979 10725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:44937 every 8 connection(s)
I20260805 23:19:29.502957 10655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:29.505859 10726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:29.505985 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10655
I20260805 23:19:29.506137 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:29.506422 23905 external_mini_cluster.cc:1428] Setting key 4c5f805befc174823c61d16a7c3e47df
I20260805 23:19:29.512382 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: Bootstrap starting.
I20260805 23:19:29.514643 10726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: Neither blocks nor log segments found. Creating new log.
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:29.515519 10726 log.cc:824] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:29.517405 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: No bootstrap required, opened a new log
I20260805 23:19:29.520532 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:29.507846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39026 (local address 127.23.88.126:44937)
0805 23:19:29.508342 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:29.508397 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:29.508494 (+ 97us) server_negotiation.cc:407] Connection header received
0805 23:19:29.509201 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:29.509218 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:29.509545 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:29.509874 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:29.510717 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.511760 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.512666 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.513132 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.513305 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:29.515871 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:29.515876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:29.515890 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:29.515918 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:29.517704 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:29.518151 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:29.518157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:29.518169 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:29.518277 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:29.518565 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:29.518569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:29.518571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:29.518957 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:29.519177 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:29.519516 (+ 339us) server_negotiation.cc:299] Negotiation successful
0805 23:19:29.519809 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":142,"threads_started":1}
I20260805 23:19:29.521222 10726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } }
I20260805 23:19:29.521474 10726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:29.521544 10726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 798b513f63694673bfcf40fa77824a58, State: Initialized, Role: FOLLOWER
I20260805 23:19:29.522001 10726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } }
I20260805 23:19:29.522164 10726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:29.522243 10726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:29.522365 10726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:29.523474 10726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } }
I20260805 23:19:29.523809 10726 leader_election.cc:302] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 798b513f63694673bfcf40fa77824a58; no voters:
I20260805 23:19:29.524128 10726 leader_election.cc:288] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:29.524261 10732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:29.524524 10732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [term 1 LEADER]: Becoming Leader. State: Replica: 798b513f63694673bfcf40fa77824a58, State: Running, Role: LEADER
I20260805 23:19:29.524844 10732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } }
I20260805 23:19:29.525416 10726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:29.526476 10733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "798b513f63694673bfcf40fa77824a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } } }
I20260805 23:19:29.526598 10733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:29.526861 10734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 798b513f63694673bfcf40fa77824a58. Latest consensus state: current_term: 1 leader_uuid: "798b513f63694673bfcf40fa77824a58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "798b513f63694673bfcf40fa77824a58" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44937 } } }
I20260805 23:19:29.526968 10734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:29.527424 10741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:29.530305 10741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:29.536219 10741 catalog_manager.cc:1347] Generated new cluster ID: 322407257c134a99ae9ce88406bfd93a
I20260805 23:19:29.536300 10741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:29.548549 10741 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:29.549566 10741 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:29.554708 10741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: Generated new TSK 0
I20260805 23:19:29.555564 10741 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:19:29.594745 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:29.711613 10753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:29.711899 10753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:29.711990 10753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:29.716624 10753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:29.716694 10753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:29.716816 10753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:29.722869 10753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:29.724102 10753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:29.725217 10753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:29.735764 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:29.735764 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.736033 10753 server_base.cc:1034] running on GCE node
W20260805 23:19:29.735816 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.736613 10753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:29.737314 10753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:29.738508 10753 hybrid_clock.cc:648] HybridClock initialized: now 1785971969738490 us; error 31 us; skew 500 ppm
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:29.741508 10753 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:19:29.742398 10753 webserver.cc:460] Webserver started at http://127.23.88.65:34767/ using document root <none> and password file <none>
I20260805 23:19:29.742988 10753 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:29.743081 10753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:29.743345 10753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:29.745446 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "4b8d460f4496421895179fd4b126f83e"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "e1add1aef012d44151554fde980003ed"
server_key_iv: "68e6988148e3033e77821e92cc17bf97"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.746052 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "4b8d460f4496421895179fd4b126f83e"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "e1add1aef012d44151554fde980003ed"
server_key_iv: "68e6988148e3033e77821e92cc17bf97"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.749994 10753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:19:29.752592 10769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:29.753865 10753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:19:29.754015 10753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "4b8d460f4496421895179fd4b126f83e"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "e1add1aef012d44151554fde980003ed"
server_key_iv: "68e6988148e3033e77821e92cc17bf97"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.754134 10753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:29.776525 10753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:29.780705 10753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:29.780936 10753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:29.781641 10753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:29.782567 10753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:29.782639 10753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:29.782712 10753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:29.782764 10753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:29.792176 10753 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:41857
I20260805 23:19:29.792212 10897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:41857 every 8 connection(s)
I20260805 23:19:29.793155 10753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:29.802448 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10753
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:29.802568 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:29.802903 23905 external_mini_cluster.cc:1428] Setting key cb87fb84da38fe6b7b7f65f4b22a29c7
I20260805 23:19:29.806902 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:29.795189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:45789 (local address 127.23.88.126:44937)
0805 23:19:29.795351 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:29.795355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:29.796387 (+ 1032us) server_negotiation.cc:407] Connection header received
0805 23:19:29.797247 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:29.797249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:29.797295 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:29.797368 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:29.798502 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.799037 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.800239 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.800470 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.800546 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:29.803250 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:29.803253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:29.803255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:29.803282 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:29.804782 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:29.805326 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:29.805336 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:29.805337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:29.805412 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:29.805769 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:29.805772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:29.805774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:29.805973 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:29.806081 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:29.806605 (+ 524us) server_negotiation.cc:299] Negotiation successful
0805 23:19:29.806738 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:19:29.807797 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:29.795584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44937 (local address 127.23.88.65:45789)
0805 23:19:29.796248 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:29.796291 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:29.797067 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:29.797520 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:29.797526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:29.797900 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:29.798374 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:29.798388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.799216 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.799219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:29.800118 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:29.800129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:29.800602 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:29.800605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:29.800751 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:29.801419 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:29.801436 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:29.803025 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:29.804908 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:29.804915 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:29.804932 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:29.805214 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:29.805506 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:29.805510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:29.805512 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:29.805664 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:29.806082 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:29.806091 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:29.806358 (+ 267us) client_negotiation.cc:769] Sending connection context
0805 23:19:29.806562 (+ 204us) client_negotiation.cc:240] Negotiation successful
0805 23:19:29.806911 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":112,"threads_started":1}
I20260805 23:19:29.809088 10898 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44937
I20260805 23:19:29.809336 10898 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:29.809854 10898 heartbeater.cc:511] Master 127.23.88.126:44937 requested a full tablet report, sending...
I20260805 23:19:29.811507 10689 ts_manager.cc:194] Registered new tserver with Master: 4b8d460f4496421895179fd4b126f83e (127.23.88.65:41857)
I20260805 23:19:29.812623 10689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:45789
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:19:29.834327 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:29.956308 10903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:29.956617 10903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:29.956663 10903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:29.961107 10903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:29.961172 10903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:29.961233 10903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:29.966744 10903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:29.967908 10903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:29.968911 10903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:29.979137 10908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:29.979146 10909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.979265 10903 server_base.cc:1034] running on GCE node
W20260805 23:19:29.979182 10912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:29.979826 10903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:29.980600 10903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:29.981829 10903 hybrid_clock.cc:648] HybridClock initialized: now 1785971969981806 us; error 37 us; skew 500 ppm
Aug 05 23:19:29 dist-test-slave-vkjp krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:29.985031 10903 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:19:29.986043 10903 webserver.cc:460] Webserver started at http://127.23.88.66:42371/ using document root <none> and password file <none>
I20260805 23:19:29.986657 10903 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:29.986754 10903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:29.987006 10903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:29.989228 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "3e36fb01b2e84d41a1bec7deac9df507"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "550ae3a55717b3351ed75aa4d3661b83"
server_key_iv: "2c1a4c39039ff7b7393ec36b7a437d58"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.989843 10903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "3e36fb01b2e84d41a1bec7deac9df507"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "550ae3a55717b3351ed75aa4d3661b83"
server_key_iv: "2c1a4c39039ff7b7393ec36b7a437d58"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.993810 10903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:19:29.996450 10920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:29.997877 10903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:19:29.998044 10903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "3e36fb01b2e84d41a1bec7deac9df507"
format_stamp: "Formatted at 2026-08-05 23:19:29 on dist-test-slave-vkjp"
server_key: "550ae3a55717b3351ed75aa4d3661b83"
server_key_iv: "2c1a4c39039ff7b7393ec36b7a437d58"
server_key_version: "encryptionkey@0"
I20260805 23:19:29.998174 10903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:30.018749 10903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:30.022320 10903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:30.022541 10903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:30.023226 10903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:30.024091 10903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:30.024137 10903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.024200 10903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:30.024235 10903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.033775 10903 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:41485
I20260805 23:19:30.033865 11043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:41485 every 8 connection(s)
I20260805 23:19:30.034735 10903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:30.041899 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 10903
I20260805 23:19:30.042060 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:30.042404 23905 external_mini_cluster.cc:1428] Setting key 7f20c98f7d3d991f34fd708ef94c31a9
Aug 05 23:19:30 dist-test-slave-vkjp krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971969, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:30.049888 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.036810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:57859 (local address 127.23.88.126:44937)
0805 23:19:30.036985 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.036989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.038006 (+ 1017us) server_negotiation.cc:407] Connection header received
0805 23:19:30.038897 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.038900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.038961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.039043 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:30.040394 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.040931 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.042211 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.042571 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.042687 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.045779 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:30.045785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:30.045790 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:30.045827 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:30.047497 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.048152 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.048155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.048156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.048235 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.048673 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.048676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.048677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.048860 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:30.048947 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.049651 (+ 704us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.049765 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:19:30.051327 11046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.037207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44937 (local address 127.23.88.66:57859)
0805 23:19:30.037875 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.037917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.038675 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.039330 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.039337 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.039711 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:30.040233 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.040248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.041074 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.041076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.042071 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.042083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.042735 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.042738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.042937 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.043717 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:30.043733 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:30.045595 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:30.047622 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:30.047631 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:30.047650 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:30.048045 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:30.048336 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:30.048343 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:30.048348 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:30.048556 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:30.048975 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:30.048982 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:30.049361 (+ 379us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.049612 (+ 251us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.050025 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":117,"threads_started":1}
I20260805 23:19:30.052642 11044 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44937
I20260805 23:19:30.052892 11044 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
I20260805 23:19:30.053519 11044 heartbeater.cc:511] Master 127.23.88.126:44937 requested a full tablet report, sending...
I20260805 23:19:30.054800 10689 ts_manager.cc:194] Registered new tserver with Master: 3e36fb01b2e84d41a1bec7deac9df507 (127.23.88.66:41485)
I20260805 23:19:30.055474 10689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:57859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:19:30.075937 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:33687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:30.195621 11049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:30.195924 11049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:30.196012 11049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:30.200264 11049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:30.200363 11049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:30.200454 11049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:30.206067 11049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:33687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:44937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:30.207330 11049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:30.208384 11049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:30.218178 11055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:30.218416 11049 server_base.cc:1034] running on GCE node
W20260805 23:19:30.218211 11057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:30.218178 11054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:30.218981 11049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:30.219703 11049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:30.220889 11049 hybrid_clock.cc:648] HybridClock initialized: now 1785971970220856 us; error 49 us; skew 500 ppm
Aug 05 23:19:30 dist-test-slave-vkjp krb5kdc[10640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971970, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:30.223757 11049 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:19:30.224603 11049 webserver.cc:460] Webserver started at http://127.23.88.67:40635/ using document root <none> and password file <none>
I20260805 23:19:30.225206 11049 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:30.225270 11049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:30.225445 11049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:30.227663 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "20d24453059049aa990cb3760aa3c64b"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "5a7a1deb6a8a84cbd0d969ac9d4bd701"
server_key_iv: "7e4e6a32f0130630fc10ac30b9ed88c2"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.228193 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "20d24453059049aa990cb3760aa3c64b"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "5a7a1deb6a8a84cbd0d969ac9d4bd701"
server_key_iv: "7e4e6a32f0130630fc10ac30b9ed88c2"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.232023 11049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:19:30.234534 11064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.235886 11049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:30.236027 11049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "20d24453059049aa990cb3760aa3c64b"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "5a7a1deb6a8a84cbd0d969ac9d4bd701"
server_key_iv: "7e4e6a32f0130630fc10ac30b9ed88c2"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.236141 11049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:30.258420 11049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:30.261795 11049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:30.262006 11049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:30.262634 11049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:30.263672 11049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:30.263723 11049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.263765 11049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:30.263816 11049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.273027 11049 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:45301
I20260805 23:19:30.273046 11187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:45301 every 8 connection(s)
I20260805 23:19:30.274009 11049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:30.283699 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11049
I20260805 23:19:30.283886 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:30.284243 23905 external_mini_cluster.cc:1428] Setting key 705037c140a0aee1faf34386b761fd2b
Aug 05 23:19:30 dist-test-slave-vkjp krb5kdc[10640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971970, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:30.289491 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.276110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:45837 (local address 127.23.88.126:44937)
0805 23:19:30.276273 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.276284 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.277548 (+ 1264us) server_negotiation.cc:407] Connection header received
0805 23:19:30.278466 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.278500 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.278560 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.278647 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:30.280217 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.280823 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.282044 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.282277 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.282368 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.285189 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:30.285192 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:30.285194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:30.285223 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:30.286877 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.287454 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.287457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.287459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.287551 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.287907 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.287909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.287910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.288135 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:30.288256 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.289164 (+ 908us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.289340 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260805 23:19:30.290421 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.276549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:44937 (local address 127.23.88.67:45837)
0805 23:19:30.277314 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.277361 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.278180 (+ 819us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.278912 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.278920 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.279419 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:30.279996 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.280010 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.280967 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.280970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.281909 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.281919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.282445 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.282448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.282594 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.283290 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:30.283307 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:30.285000 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:30.286989 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:30.286994 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:30.287008 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:30.287352 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:30.287643 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:30.287645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:30.287646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:30.287807 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:30.288340 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:30.288347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:30.288821 (+ 474us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.289094 (+ 273us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.289504 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":138,"threads_started":1}
I20260805 23:19:30.291841 11188 heartbeater.cc:347] Connected to a master server at 127.23.88.126:44937
I20260805 23:19:30.292142 11188 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:30.292729 11188 heartbeater.cc:511] Master 127.23.88.126:44937 requested a full tablet report, sending...
I20260805 23:19:30.293865 10689 ts_manager.cc:194] Registered new tserver with Master: 20d24453059049aa990cb3760aa3c64b (127.23.88.67:45301)
I20260805 23:19:30.294552 10689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:45837
I20260805 23:19:30.298367 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:30.309141 10730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.300474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39040 (local address 127.23.88.126:44937)
0805 23:19:30.300608 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.300612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.300823 (+ 211us) server_negotiation.cc:407] Connection header received
0805 23:19:30.300930 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.300932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.300976 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.301056 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:30.301809 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.302438 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.303394 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.303619 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.303706 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.305357 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:30.305360 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:30.305362 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:30.305389 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:30.306973 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.307614 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.307617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.307619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.307712 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:30.308143 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:30.308147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:30.308148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:30.308402 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:30.308540 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.308911 (+ 371us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.309014 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260805 23:19:30.313256 10689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:30.316506 10689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:30.331750 11199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.325569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:45301 (local address 127.0.0.1:45422)
0805 23:19:30.326276 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.326291 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.326426 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.327323 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.327329 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.327355 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:30.327572 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.327582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.328747 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.328751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.330049 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.330059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.331084 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.331089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.331251 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.331303 (+ 52us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.331410 (+ 107us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.331544 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":1,"spinlock_wait_cycles":2304,"thread_start_us":127,"threads_started":1}
I20260805 23:19:30.332122 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.325366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41857 (local address 127.0.0.1:55052)
0805 23:19:30.326223 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.326282 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.326424 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.327473 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.327478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.327504 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:30.327685 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.327691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.329272 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.329275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.330499 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.330510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.331666 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.331670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.331783 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.331814 (+ 31us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.331900 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.331973 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":36,"thread_start_us":90,"threads_started":1}
I20260805 23:19:30.332693 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.325366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:41485 (local address 127.0.0.1:37808)
0805 23:19:30.326711 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.326727 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.326840 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.327645 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.327651 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.327674 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:30.327812 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.327819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.329272 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.329275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.330933 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.330941 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.332285 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.332289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.332405 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.332431 (+ 26us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.332501 (+ 70us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.332564 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1237,"mutex_wait_us":245,"thread_start_us":103,"threads_started":1}
I20260805 23:19:30.332702 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.325542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55052 (local address 127.23.88.65:41857)
0805 23:19:30.327012 (+ 1470us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.327018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.327040 (+ 22us) server_negotiation.cc:407] Connection header received
0805 23:19:30.327138 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.327144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.327314 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.327811 (+ 497us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:30.327964 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.329126 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.330643 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.331524 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.331649 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.331821 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.332458 (+ 637us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.332570 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1362,"thread_start_us":92,"threads_started":1}
I20260805 23:19:30.332697 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.326207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45422 (local address 127.23.88.67:45301)
0805 23:19:30.326770 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.326783 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.326806 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:19:30.326900 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.326905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.327153 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.327325 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:30.327733 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.328583 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.330179 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.330924 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.332133 (+ 1209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.332331 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.332392 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.332525 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":179,"threads_started":1}
I20260805 23:19:30.333424 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.325909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37808 (local address 127.23.88.66:41485)
0805 23:19:30.326770 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.326783 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.326806 (+ 23us) server_negotiation.cc:407] Connection header received
0805 23:19:30.327037 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.327045 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.327303 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.327461 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:30.327949 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.329126 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.331051 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.331720 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.331827 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.331995 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.333202 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.333303 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":106,"threads_started":1}
I20260805 23:19:30.335913 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 1144a87205274f3baf0153be872edf4d (DEFAULT_TABLE table=test-table [id=ecb8509f2a82433caf4af16f6112f28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:30.335870 11117 tablet_service.cc:1467] Processing CreateTablet for tablet 1144a87205274f3baf0153be872edf4d (DEFAULT_TABLE table=test-table [id=ecb8509f2a82433caf4af16f6112f28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:30.335870 10823 tablet_service.cc:1467] Processing CreateTablet for tablet 1144a87205274f3baf0153be872edf4d (DEFAULT_TABLE table=test-table [id=ecb8509f2a82433caf4af16f6112f28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:30.336905 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1144a87205274f3baf0153be872edf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:30.336936 11117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1144a87205274f3baf0153be872edf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:30.336928 10823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1144a87205274f3baf0153be872edf4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:30.343922 11206 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Bootstrap starting.
I20260805 23:19:30.344841 11208 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Bootstrap starting.
I20260805 23:19:30.345158 11207 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Bootstrap starting.
I20260805 23:19:30.346915 11207 tablet_bootstrap.cc:654] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:30.347031 11208 tablet_bootstrap.cc:654] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:30.347131 11206 tablet_bootstrap.cc:654] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:30.347784 11207 log.cc:824] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:30.348026 11206 log.cc:824] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:30.348251 11208 log.cc:824] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:30.349910 11207 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: No bootstrap required, opened a new log
I20260805 23:19:30.350116 11207 ts_tablet_manager.cc:1397] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260805 23:19:30.350163 11206 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: No bootstrap required, opened a new log
I20260805 23:19:30.350364 11206 ts_tablet_manager.cc:1397] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260805 23:19:30.351513 11208 tablet_bootstrap.cc:492] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: No bootstrap required, opened a new log
I20260805 23:19:30.351703 11208 ts_tablet_manager.cc:1397] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260805 23:19:30.353999 11207 raft_consensus.cc:348] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.354257 11207 raft_consensus.cc:374] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:30.354321 11207 raft_consensus.cc:729] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b8d460f4496421895179fd4b126f83e, State: Initialized, Role: FOLLOWER
I20260805 23:19:30.354254 11206 raft_consensus.cc:348] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.354529 11206 raft_consensus.cc:374] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:30.354588 11206 raft_consensus.cc:729] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e36fb01b2e84d41a1bec7deac9df507, State: Initialized, Role: FOLLOWER
I20260805 23:19:30.354905 11207 consensus_queue.cc:260] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.355024 11206 consensus_queue.cc:260] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.355655 10898 heartbeater.cc:503] Master 127.23.88.126:44937 was elected leader, sending a full tablet report...
I20260805 23:19:30.355808 11208 raft_consensus.cc:348] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.356053 11207 ts_tablet_manager.cc:1428] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260805 23:19:30.356063 11208 raft_consensus.cc:374] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:30.356114 11208 raft_consensus.cc:729] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20d24453059049aa990cb3760aa3c64b, State: Initialized, Role: FOLLOWER
I20260805 23:19:30.356534 11044 heartbeater.cc:503] Master 127.23.88.126:44937 was elected leader, sending a full tablet report...
I20260805 23:19:30.356518 11208 consensus_queue.cc:260] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.357514 11208 ts_tablet_manager.cc:1428] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260805 23:19:30.357513 11206 ts_tablet_manager.cc:1428] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:19:30.357837 11188 heartbeater.cc:503] Master 127.23.88.126:44937 was elected leader, sending a full tablet report...
W20260805 23:19:30.525113 11189 tablet.cc:2369] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:30.530157 11215 raft_consensus.cc:482] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:30.530393 11215 raft_consensus.cc:504] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.531615 11215 leader_election.cc:288] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b8d460f4496421895179fd4b126f83e (127.23.88.65:41857), 3e36fb01b2e84d41a1bec7deac9df507 (127.23.88.66:41485)
I20260805 23:19:30.535531 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.531949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41857 (local address 127.23.88.67:37591)
0805 23:19:30.532090 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.532108 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.532220 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.532500 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.532502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.532517 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:30.532619 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.532623 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.533572 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.533577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.534354 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.534362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.535054 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.535058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.535207 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.535226 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.535312 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.535387 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":2}
I20260805 23:19:30.535531 11200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.532101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:37591 (local address 127.23.88.65:41857)
0805 23:19:30.532244 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.532250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.532269 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:19:30.532338 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.532341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.532394 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.532464 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:30.532742 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.533461 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.534487 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.534917 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.535058 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.535132 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.535330 (+ 198us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.535407 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260805 23:19:30.535987 11045 tablet.cc:2369] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:30.536348 10846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1144a87205274f3baf0153be872edf4d" candidate_uuid: "20d24453059049aa990cb3760aa3c64b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8d460f4496421895179fd4b126f83e" is_pre_election: true
I20260805 23:19:30.536494 11216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.531949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:41485 (local address 127.23.88.67:41145)
0805 23:19:30.532927 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.532949 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:30.533046 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:30.533349 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:30.533351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:30.533369 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:30.533473 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.533479 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.534272 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.534276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.535134 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:30.535150 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.536075 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.536079 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:30.536169 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.536204 (+ 35us) client_negotiation.cc:769] Sending connection context
0805 23:19:30.536290 (+ 86us) client_negotiation.cc:240] Negotiation successful
0805 23:19:30.536355 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":583,"thread_start_us":159,"threads_started":1}
I20260805 23:19:30.536541 11203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.532080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:41145 (local address 127.23.88.66:41485)
0805 23:19:30.532233 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.532239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.533015 (+ 776us) server_negotiation.cc:407] Connection header received
0805 23:19:30.533122 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.533124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.533174 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.533252 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:30.533629 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.534144 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.535265 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.535797 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.535917 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.535973 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.536324 (+ 351us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.536421 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:19:30.536701 10846 raft_consensus.cc:2454] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20d24453059049aa990cb3760aa3c64b in term 0.
I20260805 23:19:30.537180 10992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1144a87205274f3baf0153be872edf4d" candidate_uuid: "20d24453059049aa990cb3760aa3c64b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e36fb01b2e84d41a1bec7deac9df507" is_pre_election: true
I20260805 23:19:30.537245 11068 leader_election.cc:302] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20d24453059049aa990cb3760aa3c64b, 4b8d460f4496421895179fd4b126f83e; no voters:
I20260805 23:19:30.537487 10992 raft_consensus.cc:2454] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20d24453059049aa990cb3760aa3c64b in term 0.
I20260805 23:19:30.537532 11215 raft_consensus.cc:2781] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:30.537647 11215 raft_consensus.cc:482] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:30.537724 11215 raft_consensus.cc:3037] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:30.538815 11215 raft_consensus.cc:504] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
I20260805 23:19:30.539237 11215 leader_election.cc:288] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [CANDIDATE]: Term 1 election: Requested vote from peers 4b8d460f4496421895179fd4b126f83e (127.23.88.65:41857), 3e36fb01b2e84d41a1bec7deac9df507 (127.23.88.66:41485)
I20260805 23:19:30.539621 10992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1144a87205274f3baf0153be872edf4d" candidate_uuid: "20d24453059049aa990cb3760aa3c64b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e36fb01b2e84d41a1bec7deac9df507"
I20260805 23:19:30.539659 10846 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1144a87205274f3baf0153be872edf4d" candidate_uuid: "20d24453059049aa990cb3760aa3c64b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b8d460f4496421895179fd4b126f83e"
I20260805 23:19:30.539759 10992 raft_consensus.cc:3037] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:30.539836 10846 raft_consensus.cc:3037] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:30.541178 10846 raft_consensus.cc:2454] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20d24453059049aa990cb3760aa3c64b in term 1.
I20260805 23:19:30.541178 10992 raft_consensus.cc:2454] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20d24453059049aa990cb3760aa3c64b in term 1.
I20260805 23:19:30.541524 11068 leader_election.cc:302] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20d24453059049aa990cb3760aa3c64b, 4b8d460f4496421895179fd4b126f83e; no voters:
I20260805 23:19:30.541740 11215 raft_consensus.cc:2781] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:30.542016 11215 raft_consensus.cc:686] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 1 LEADER]: Becoming Leader. State: Replica: 20d24453059049aa990cb3760aa3c64b, State: Running, Role: LEADER
I20260805 23:19:30.542385 11215 consensus_queue.cc:237] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } }
W20260805 23:19:30.544615 10899 tablet.cc:2369] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:30.545920 10687 catalog_manager.cc:5495] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b reported cstate change: term changed from 0 to 1, leader changed from <none> to 20d24453059049aa990cb3760aa3c64b (127.23.88.67). New cstate: current_term: 1 leader_uuid: "20d24453059049aa990cb3760aa3c64b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20d24453059049aa990cb3760aa3c64b" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 45301 } health_report { overall_health: HEALTHY } } }
I20260805 23:19:30.571534 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.567832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.23.88.67:45301)
0805 23:19:30.567993 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.567996 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.568009 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:19:30.568082 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.568085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.568136 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.568224 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:30.568503 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.569004 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.569908 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.570120 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.570197 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.570546 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:30.570969 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:30.571139 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.571307 (+ 168us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.571373 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260805 23:19:30.577704 10992 raft_consensus.cc:1261] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 1 FOLLOWER]: Refusing update from remote peer 20d24453059049aa990cb3760aa3c64b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:30.577704 10846 raft_consensus.cc:1261] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 1 FOLLOWER]: Refusing update from remote peer 20d24453059049aa990cb3760aa3c64b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:30.578402 11215 consensus_queue.cc:1035] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e36fb01b2e84d41a1bec7deac9df507" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 41485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:30.578670 11217 consensus_queue.cc:1035] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b8d460f4496421895179fd4b126f83e" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 41857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:30.588410 11221 mvcc.cc:205] Tried to move back new op lower bound from 7315341191478546432 to 7315341191344459776. Current Snapshot: MvccSnapshot[applied={T|T < 7315341191478546432}]
I20260805 23:19:30.591941 11222 mvcc.cc:205] Tried to move back new op lower bound from 7315341191478546432 to 7315341191344459776. Current Snapshot: MvccSnapshot[applied={T|T < 7315341191478546432 or (T in {7315341191478546432})}]
I20260805 23:19:30.596297 10687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:30.596462 10687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39040:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:30.599176 10687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 798b513f63694673bfcf40fa77824a58: Sending DeleteTablet for 3 replicas of tablet 1144a87205274f3baf0153be872edf4d
I20260805 23:19:30.600194 10823 tablet_service.cc:1514] Processing DeleteTablet for tablet 1144a87205274f3baf0153be872edf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:55052
I20260805 23:19:30.600363 11117 tablet_service.cc:1514] Processing DeleteTablet for tablet 1144a87205274f3baf0153be872edf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:45422
I20260805 23:19:30.600795 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet 1144a87205274f3baf0153be872edf4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:30 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:37808
I20260805 23:19:30.600857 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10753
I20260805 23:19:30.600965 11229 tablet_replica.cc:330] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e: stopping tablet replica
I20260805 23:19:30.601337 11229 raft_consensus.cc:2229] T 1144a87205274f3baf0153be872edf4d P 4b8d460f4496421895179fd4b126f83e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:30.601590 11230 tablet_replica.cc:330] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: stopping tablet replica
I20260805 23:19:30.601990 11230 raft_consensus.cc:2229] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:19:30.602394 11230 raft_consensus.cc:2258] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:30.605245 11231 tablet_replica.cc:330] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: stopping tablet replica
I20260805 23:19:30.605571 11231 raft_consensus.cc:2229] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:30.605903 11231 raft_consensus.cc:2258] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:30.607312 11230 ts_tablet_manager.cc:1905] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:30.607574 11231 ts_tablet_manager.cc:1905] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:30.610332 11231 ts_tablet_manager.cc:1918] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:30.610335 11230 ts_tablet_manager.cc:1918] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:30.610446 11230 log.cc:1196] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/1144a87205274f3baf0153be872edf4d
I20260805 23:19:30.610599 11231 log.cc:1196] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/1144a87205274f3baf0153be872edf4d
W20260805 23:19:30.610927 10674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:41857 (error 108)
I20260805 23:19:30.611048 11230 ts_tablet_manager.cc:1939] T 1144a87205274f3baf0153be872edf4d P 20d24453059049aa990cb3760aa3c64b: Deleting consensus metadata
I20260805 23:19:30.611177 11231 ts_tablet_manager.cc:1939] T 1144a87205274f3baf0153be872edf4d P 3e36fb01b2e84d41a1bec7deac9df507: Deleting consensus metadata
I20260805 23:19:30.611866 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10903
I20260805 23:19:30.615032 10673 catalog_manager.cc:4841] TS 20d24453059049aa990cb3760aa3c64b (127.23.88.67:45301): tablet 1144a87205274f3baf0153be872edf4d (table test-table [id=ecb8509f2a82433caf4af16f6112f28b]) successfully deleted
I20260805 23:19:30.619064 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11049
I20260805 23:19:30.622284 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.621571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:41857 (local address 127.0.0.1:51214)
0805 23:19:30.622065 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.622176 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:41857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375}
I20260805 23:19:30.622539 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.621980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:41485 (local address 127.0.0.1:45680)
0805 23:19:30.622442 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:30.622491 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:41485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409}
W20260805 23:19:30.622736 10671 catalog_manager.cc:4584] TS 3e36fb01b2e84d41a1bec7deac9df507 (127.23.88.66:41485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1144a87205274f3baf0153be872edf4d: Network error: Client connection negotiation failed: client connection to 127.23.88.66:41485: connect: Connection refused (error 111)
I20260805 23:19:30.628230 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 10655
2026-08-05T23:19:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2519 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-05T23:19:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:30Z Disabled control of system clock
I20260805 23:19:30.661281 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41603
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41603
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:30.775097 11237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:30.775399 11237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:30.775437 11237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:30.779692 11237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:30.779764 11237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:30.779780 11237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:30.779798 11237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:30.779814 11237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:30.784816 11237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41603
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:30.786005 11237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:30.787545 11237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:30.796283 11243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:30.796307 11246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:30.796307 11242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:30.796667 11237 server_base.cc:1034] running on GCE node
I20260805 23:19:30.797358 11237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:30.798542 11237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:30.799728 11237 hybrid_clock.cc:648] HybridClock initialized: now 1785971970799704 us; error 48 us; skew 500 ppm
I20260805 23:19:30.801714 11237 webserver.cc:460] Webserver started at http://127.23.88.126:42255/ using document root <none> and password file <none>
I20260805 23:19:30.802320 11237 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:30.802420 11237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:30.802661 11237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:30.804857 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "e133832ee3d44d46a672f32dfdd4be7f"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "b9212c221e575009aaeae2947ec0f1f9"
server_key_iv: "c920dc14cf0bc4088f746240d77344c6"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.805454 11237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "e133832ee3d44d46a672f32dfdd4be7f"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "b9212c221e575009aaeae2947ec0f1f9"
server_key_iv: "c920dc14cf0bc4088f746240d77344c6"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.809376 11237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260805 23:19:30.811971 11252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:30.813184 11237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:30.813329 11237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "e133832ee3d44d46a672f32dfdd4be7f"
format_stamp: "Formatted at 2026-08-05 23:19:30 on dist-test-slave-vkjp"
server_key: "b9212c221e575009aaeae2947ec0f1f9"
server_key_iv: "c920dc14cf0bc4088f746240d77344c6"
server_key_version: "encryptionkey@0"
I20260805 23:19:30.813458 11237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:30.827190 11237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:30.827973 11237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:30.828188 11237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:30.835834 11237 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41603
I20260805 23:19:30.835852 11308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41603 every 8 connection(s)
I20260805 23:19:30.836813 11237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:30.837770 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11237
I20260805 23:19:30.837916 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:30.838233 23905 external_mini_cluster.cc:1428] Setting key 930b0608347d7a2380c0c8be54eadbd3
I20260805 23:19:30.840415 11309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:30.847028 11309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: Bootstrap starting.
I20260805 23:19:30.847725 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:30.839590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51422 (local address 127.23.88.126:41603)
0805 23:19:30.840202 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:30.840217 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:30.840252 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:19:30.840947 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:30.840964 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:30.841348 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:30.841629 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:30.842034 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.843022 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.844252 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:30.844706 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:30.844827 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:30.845793 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:30.845798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:30.845811 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:30.846191 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:30.846245 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:30.846263 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:30.846336 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:30.846566 (+ 230us) server_negotiation.cc:299] Negotiation successful
0805 23:19:30.846857 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":148,"threads_started":1}
I20260805 23:19:30.849575 11309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:30.850441 11309 log.cc:824] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:30.852497 11309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: No bootstrap required, opened a new log
I20260805 23:19:30.856993 11309 raft_consensus.cc:348] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } }
I20260805 23:19:30.857267 11309 raft_consensus.cc:374] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:30.857317 11309 raft_consensus.cc:729] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e133832ee3d44d46a672f32dfdd4be7f, State: Initialized, Role: FOLLOWER
I20260805 23:19:30.857718 11309 consensus_queue.cc:260] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } }
I20260805 23:19:30.857843 11309 raft_consensus.cc:388] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:30.857894 11309 raft_consensus.cc:482] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:30.857972 11309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:30.859000 11309 raft_consensus.cc:504] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } }
I20260805 23:19:30.859423 11309 leader_election.cc:302] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e133832ee3d44d46a672f32dfdd4be7f; no voters:
I20260805 23:19:30.859733 11309 leader_election.cc:288] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:30.859866 11315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:30.860179 11315 raft_consensus.cc:686] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [term 1 LEADER]: Becoming Leader. State: Replica: e133832ee3d44d46a672f32dfdd4be7f, State: Running, Role: LEADER
I20260805 23:19:30.860517 11315 consensus_queue.cc:237] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } }
I20260805 23:19:30.861193 11309 sys_catalog.cc:564] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:30.862303 11317 sys_catalog.cc:455] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e133832ee3d44d46a672f32dfdd4be7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } } }
I20260805 23:19:30.862468 11317 sys_catalog.cc:458] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:30.862831 11325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:30.863166 11318 sys_catalog.cc:455] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e133832ee3d44d46a672f32dfdd4be7f. Latest consensus state: current_term: 1 leader_uuid: "e133832ee3d44d46a672f32dfdd4be7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e133832ee3d44d46a672f32dfdd4be7f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41603 } } }
I20260805 23:19:30.863310 11318 sys_catalog.cc:458] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:30.866173 11325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:30.871961 11325 catalog_manager.cc:1347] Generated new cluster ID: 12bdec33fd36498abf1ea2ca6815b0b5
I20260805 23:19:30.872033 11325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:30.879938 11325 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:30.880897 11325 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:30.890919 11325 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: Generated new TSK 0
I20260805 23:19:30.891665 11325 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:30.902683 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--builtin_ntp_servers=127.23.88.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:31.019016 11335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:31.019362 11335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:31.019403 11335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:31.023609 11335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:31.023679 11335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:31.023738 11335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:31.028717 11335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11335
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:31.029951 11335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:31.031598 11335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:31.041891 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:31.042045 11335 server_base.cc:1034] running on GCE node
W20260805 23:19:31.041904 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:31.041891 11340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:31.042685 11335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:31.043480 11335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:31.044679 11335 hybrid_clock.cc:648] HybridClock initialized: now 1785971971044663 us; error 31 us; skew 500 ppm
I20260805 23:19:31.052002 11335 webserver.cc:460] Webserver started at http://127.23.88.65:41063/ using document root <none> and password file <none>
I20260805 23:19:31.052668 11335 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:31.052773 11335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:31.053030 11335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:31.055320 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "ea21b9d3aafd43018a38584aafceecb8"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "7142092a65870d754a442223c9f8a225"
server_key_iv: "6c4dd127d6f3412c9e044a42c43b95bb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.055931 11335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "ea21b9d3aafd43018a38584aafceecb8"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "7142092a65870d754a442223c9f8a225"
server_key_iv: "6c4dd127d6f3412c9e044a42c43b95bb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.059973 11335 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:31.062726 11350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.064177 11335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:31.064311 11335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "ea21b9d3aafd43018a38584aafceecb8"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "7142092a65870d754a442223c9f8a225"
server_key_iv: "6c4dd127d6f3412c9e044a42c43b95bb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.064435 11335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:31.088784 11335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:31.089557 11335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:31.089775 11335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:31.090442 11335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:31.091468 11335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:31.091518 11335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.091591 11335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:31.091622 11335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.100739 11335 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:45683
I20260805 23:19:31.100831 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:45683 every 8 connection(s)
I20260805 23:19:31.101748 11335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:31.110335 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11335
I20260805 23:19:31.110522 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:31.110816 23905 external_mini_cluster.cc:1428] Setting key 5b6823004fad275f606e0809e3d2880f
I20260805 23:19:31.111642 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.103653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:54213 (local address 127.23.88.126:41603)
0805 23:19:31.103821 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.103826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.104929 (+ 1103us) server_negotiation.cc:407] Connection header received
0805 23:19:31.106354 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.106357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.106410 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.106488 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:31.107554 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.108128 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.109336 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.109563 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.109645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.111035 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:31.111039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:31.111042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:31.111058 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:31.111104 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:31.111115 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:31.111194 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.111375 (+ 181us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.111483 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260805 23:19:31.112879 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.104114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41603 (local address 127.23.88.65:54213)
0805 23:19:31.104772 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.104815 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.106141 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.106654 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.106660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.106743 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:31.107328 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.107345 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.108262 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.108264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.109203 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.109213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.109738 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.109741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.109898 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.110704 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:31.110705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:31.110766 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:31.110871 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:31.111204 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:31.111213 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:31.111220 (+ 7us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.111358 (+ 138us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.111893 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
I20260805 23:19:31.113147 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--builtin_ntp_servers=127.23.88.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:31.114194 11475 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41603
I20260805 23:19:31.114465 11475 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:31.115072 11475 heartbeater.cc:511] Master 127.23.88.126:41603 requested a full tablet report, sending...
I20260805 23:19:31.116786 11269 ts_manager.cc:194] Registered new tserver with Master: ea21b9d3aafd43018a38584aafceecb8 (127.23.88.65:45683)
I20260805 23:19:31.118029 11269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.65:54213
W20260805 23:19:31.229923 11478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:31.230206 11478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:31.230248 11478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:31.234486 11478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:31.234563 11478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:31.234622 11478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:31.239547 11478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:31.240747 11478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:31.242177 11478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:31.252085 11484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:31.252077 11486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:31.252077 11483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:31.252373 11478 server_base.cc:1034] running on GCE node
I20260805 23:19:31.252910 11478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:31.253648 11478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:31.254842 11478 hybrid_clock.cc:648] HybridClock initialized: now 1785971971254835 us; error 35 us; skew 500 ppm
I20260805 23:19:31.257244 11478 webserver.cc:460] Webserver started at http://127.23.88.66:44259/ using document root <none> and password file <none>
I20260805 23:19:31.257874 11478 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:31.257982 11478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:31.258215 11478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:31.260455 11478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "c9221f2f93f54aae92f8ec01268edd78"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "131a4b72f877597cff700e4be48cd570"
server_key_iv: "5d50acfdb4dcdc64c272aee2a5ddb8eb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.261092 11478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "c9221f2f93f54aae92f8ec01268edd78"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "131a4b72f877597cff700e4be48cd570"
server_key_iv: "5d50acfdb4dcdc64c272aee2a5ddb8eb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.265200 11478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:31.267897 11492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.269344 11478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:31.269510 11478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "c9221f2f93f54aae92f8ec01268edd78"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "131a4b72f877597cff700e4be48cd570"
server_key_iv: "5d50acfdb4dcdc64c272aee2a5ddb8eb"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.269640 11478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:31.284736 11478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:31.285521 11478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:31.285753 11478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:31.286414 11478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:31.287472 11478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:31.287552 11478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.287631 11478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:31.287688 11478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.297286 11478 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:42677
I20260805 23:19:31.297317 11614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:42677 every 8 connection(s)
I20260805 23:19:31.298306 11478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:31.300877 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11478
I20260805 23:19:31.301038 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:31.301427 23905 external_mini_cluster.cc:1428] Setting key 39306158d25d7356d55a2461cea6ff5a
I20260805 23:19:31.303843 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--builtin_ntp_servers=127.23.88.84:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:31.309856 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.300328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:57095 (local address 127.23.88.126:41603)
0805 23:19:31.300523 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.300529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.301875 (+ 1346us) server_negotiation.cc:407] Connection header received
0805 23:19:31.303666 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.303669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.303720 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.303788 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:31.305046 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.305957 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.307209 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.307673 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.307812 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.309150 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:31.309157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:31.309164 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:31.309185 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:31.309218 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:31.309233 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:31.309330 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.309497 (+ 167us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.309595 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:19:31.310595 11617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.300773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41603 (local address 127.23.88.66:57095)
0805 23:19:31.301662 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.301722 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.303424 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.304021 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.304032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.304141 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:31.304854 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.304876 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.306109 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.306112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.307028 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.307037 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.307869 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.307873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.308029 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.308859 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:31.308861 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:31.308903 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:31.308970 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:31.309345 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:31.309353 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:31.309357 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.309455 (+ 98us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.309689 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":157,"threads_started":1}
I20260805 23:19:31.311911 11615 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41603
I20260805 23:19:31.312175 11615 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:31.312809 11615 heartbeater.cc:511] Master 127.23.88.126:41603 requested a full tablet report, sending...
I20260805 23:19:31.313925 11269 ts_manager.cc:194] Registered new tserver with Master: c9221f2f93f54aae92f8ec01268edd78 (127.23.88.66:42677)
I20260805 23:19:31.314522 11269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.66:57095
W20260805 23:19:31.422075 11618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:31.422353 11618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:31.422426 11618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:31.426465 11618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:31.426527 11618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:31.426620 11618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:31.432143 11618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:41603
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11618
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:31.433280 11618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:31.434693 11618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:31.444633 11624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:31.444651 11625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:31.444706 11618 server_base.cc:1034] running on GCE node
W20260805 23:19:31.444657 11627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:31.445430 11618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:31.446067 11618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:31.447228 11618 hybrid_clock.cc:648] HybridClock initialized: now 1785971971447199 us; error 48 us; skew 500 ppm
I20260805 23:19:31.449262 11618 webserver.cc:460] Webserver started at http://127.23.88.67:44981/ using document root <none> and password file <none>
I20260805 23:19:31.449856 11618 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:31.449923 11618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:31.450147 11618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:31.452404 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "9f7fd8acc1e5422db1d622af5218dd37"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "d6c1d48d413769cbb8674852a83e5040"
server_key_iv: "1c11085169c0077e1ac63043ee2af697"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.453014 11618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "9f7fd8acc1e5422db1d622af5218dd37"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "d6c1d48d413769cbb8674852a83e5040"
server_key_iv: "1c11085169c0077e1ac63043ee2af697"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.456770 11618 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260805 23:19:31.459319 11635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.460554 11618 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:31.460697 11618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "9f7fd8acc1e5422db1d622af5218dd37"
format_stamp: "Formatted at 2026-08-05 23:19:31 on dist-test-slave-vkjp"
server_key: "d6c1d48d413769cbb8674852a83e5040"
server_key_iv: "1c11085169c0077e1ac63043ee2af697"
server_key_version: "encryptionkey@0"
I20260805 23:19:31.460824 11618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:31.481828 11618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:31.482590 11618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:31.482800 11618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:31.483515 11618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:31.484501 11618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:31.484572 11618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.484645 11618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:31.484686 11618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:31.494287 11618 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:39285
I20260805 23:19:31.494310 11762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:39285 every 8 connection(s)
I20260805 23:19:31.495250 11618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:31.501613 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11618
I20260805 23:19:31.501766 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:31.502054 23905 external_mini_cluster.cc:1428] Setting key fcebfea76b1d43e1924d627882147a6a
I20260805 23:19:31.505249 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.497218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:42689 (local address 127.23.88.126:41603)
0805 23:19:31.497392 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.497396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.498493 (+ 1097us) server_negotiation.cc:407] Connection header received
0805 23:19:31.500030 (+ 1537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.500034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.500097 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.500207 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:31.501139 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.501677 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.503180 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.503460 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.503570 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.504822 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:31.504826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:31.504828 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:31.504840 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:31.504861 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:31.504869 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:31.504930 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.505081 (+ 151us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.505136 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260805 23:19:31.506103 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.497596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:41603 (local address 127.23.88.67:42689)
0805 23:19:31.498261 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.498364 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.499804 (+ 1440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.500391 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.500398 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.500486 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:31.500972 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.500997 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.501824 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.501830 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.503000 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.503012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.503628 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.503632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.503838 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.504551 (+ 713us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:31.504553 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:31.504596 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:31.504660 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:31.504954 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:31.504961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:31.504966 (+ 5us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.505060 (+ 94us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.505288 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":112,"threads_started":1}
I20260805 23:19:31.507267 11763 heartbeater.cc:347] Connected to a master server at 127.23.88.126:41603
I20260805 23:19:31.507550 11763 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:31.508139 11763 heartbeater.cc:511] Master 127.23.88.126:41603 requested a full tablet report, sending...
I20260805 23:19:31.509248 11269 ts_manager.cc:194] Registered new tserver with Master: 9f7fd8acc1e5422db1d622af5218dd37 (127.23.88.67:39285)
I20260805 23:19:31.509807 11269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.67:42689
I20260805 23:19:31.516577 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:31.522557 11313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.518605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51438 (local address 127.23.88.126:41603)
0805 23:19:31.518728 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.518732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.518948 (+ 216us) server_negotiation.cc:407] Connection header received
0805 23:19:31.519030 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.519032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.519149 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.519223 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:31.519546 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.520070 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.520904 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.521142 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.521224 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.522116 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:31.522119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:31.522121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:31.522132 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:31.522147 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:31.522154 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:31.522248 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.522407 (+ 159us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.522439 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:19:31.525772 11269 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:31.529219 11269 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:31.544553 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.539199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37924 (local address 127.23.88.67:39285)
0805 23:19:31.539573 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.539584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.539712 (+ 128us) server_negotiation.cc:407] Connection header received
0805 23:19:31.539799 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.539804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.539997 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.540140 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:31.540402 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541532 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.543277 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.543824 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.543951 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544158 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.544244 (+ 86us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.544354 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":151,"threads_started":1}
I20260805 23:19:31.544569 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.537940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:42677 (local address 127.0.0.1:58156)
0805 23:19:31.538373 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.538440 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.538570 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.539569 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.539576 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.539605 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:31.539807 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.539815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541521 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.541525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.543027 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.543038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.544069 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.544073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.544159 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544182 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.544236 (+ 54us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.544397 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":53,"thread_start_us":125,"threads_started":1}
I20260805 23:19:31.544549 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.538754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39285 (local address 127.0.0.1:37924)
0805 23:19:31.539601 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.539614 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.539702 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.540145 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.540148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.540164 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:31.540277 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.540283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541625 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.541628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.543162 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.543171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.543950 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.543954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.544068 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544106 (+ 38us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.544217 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.544397 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":354,"thread_start_us":93,"threads_started":1}
I20260805 23:19:31.544945 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.538847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.23.88.65:45683)
0805 23:19:31.539295 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.539307 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.539567 (+ 260us) server_negotiation.cc:407] Connection header received
0805 23:19:31.539818 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.539823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.539980 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.540101 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:31.540509 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541294 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.542957 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.543853 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.543994 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544194 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.544500 (+ 306us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.544609 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":123,"threads_started":1}
I20260805 23:19:31.545148 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.538137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.23.88.66:42677)
0805 23:19:31.538959 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.538975 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.539004 (+ 29us) server_negotiation.cc:407] Connection header received
0805 23:19:31.539109 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.539115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.539347 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.539964 (+ 617us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:31.540122 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541355 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.543170 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.543934 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.544634 (+ 700us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544815 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.544866 (+ 51us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.544969 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":128,"threads_started":1}
I20260805 23:19:31.545367 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.538313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45683 (local address 127.0.0.1:50156)
0805 23:19:31.539458 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.539473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.539650 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.540270 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.540273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.540288 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:31.540382 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.540390 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.541542 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.541546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.542824 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.542835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.544253 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.544256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.544345 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.544364 (+ 19us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.545164 (+ 800us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.545245 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1079,"mutex_wait_us":230,"thread_start_us":118,"threads_started":1}
I20260805 23:19:31.548099 11544 tablet_service.cc:1467] Processing CreateTablet for tablet 991936956f044b558ef5fa056b9e89fa (DEFAULT_TABLE table=test-table [id=253c60ababde4790be055abd3ef25d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:31.548173 11401 tablet_service.cc:1467] Processing CreateTablet for tablet 991936956f044b558ef5fa056b9e89fa (DEFAULT_TABLE table=test-table [id=253c60ababde4790be055abd3ef25d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:31.548177 11690 tablet_service.cc:1467] Processing CreateTablet for tablet 991936956f044b558ef5fa056b9e89fa (DEFAULT_TABLE table=test-table [id=253c60ababde4790be055abd3ef25d77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:31.549224 11690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991936956f044b558ef5fa056b9e89fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:31.549227 11544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991936956f044b558ef5fa056b9e89fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:31.549307 11401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 991936956f044b558ef5fa056b9e89fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:31.556058 11782 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Bootstrap starting.
I20260805 23:19:31.557140 11784 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Bootstrap starting.
I20260805 23:19:31.557533 11783 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Bootstrap starting.
I20260805 23:19:31.558470 11782 tablet_bootstrap.cc:654] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:31.559278 11782 log.cc:824] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:31.559316 11784 tablet_bootstrap.cc:654] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:31.559442 11783 tablet_bootstrap.cc:654] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:31.560161 11784 log.cc:824] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:31.560161 11783 log.cc:824] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:31.561787 11782 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: No bootstrap required, opened a new log
I20260805 23:19:31.562016 11782 ts_tablet_manager.cc:1397] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260805 23:19:31.562194 11784 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: No bootstrap required, opened a new log
I20260805 23:19:31.562206 11783 tablet_bootstrap.cc:492] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: No bootstrap required, opened a new log
I20260805 23:19:31.562412 11783 ts_tablet_manager.cc:1397] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:31.562426 11784 ts_tablet_manager.cc:1397] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:31.566125 11784 raft_consensus.cc:348] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.566449 11784 raft_consensus.cc:374] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:31.566516 11784 raft_consensus.cc:729] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f7fd8acc1e5422db1d622af5218dd37, State: Initialized, Role: FOLLOWER
I20260805 23:19:31.566744 11782 raft_consensus.cc:348] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.566744 11783 raft_consensus.cc:348] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.566994 11782 raft_consensus.cc:374] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:31.567013 11783 raft_consensus.cc:374] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:31.567050 11782 raft_consensus.cc:729] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9221f2f93f54aae92f8ec01268edd78, State: Initialized, Role: FOLLOWER
I20260805 23:19:31.566998 11784 consensus_queue.cc:260] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.567059 11783 raft_consensus.cc:729] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea21b9d3aafd43018a38584aafceecb8, State: Initialized, Role: FOLLOWER
I20260805 23:19:31.567540 11782 consensus_queue.cc:260] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.567543 11783 consensus_queue.cc:260] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.568221 11784 ts_tablet_manager.cc:1428] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:19:31.568502 11782 ts_tablet_manager.cc:1428] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260805 23:19:31.568547 11783 ts_tablet_manager.cc:1428] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260805 23:19:31.568640 11763 heartbeater.cc:503] Master 127.23.88.126:41603 was elected leader, sending a full tablet report...
I20260805 23:19:31.568836 11475 heartbeater.cc:503] Master 127.23.88.126:41603 was elected leader, sending a full tablet report...
I20260805 23:19:31.568837 11615 heartbeater.cc:503] Master 127.23.88.126:41603 was elected leader, sending a full tablet report...
W20260805 23:19:31.602998 11476 tablet.cc:2369] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:19:31.746304 11764 tablet.cc:2369] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:19:31.799544 11616 tablet.cc:2369] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:31.805752 11790 raft_consensus.cc:482] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:31.806010 11790 raft_consensus.cc:504] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.807365 11790 leader_election.cc:288] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f7fd8acc1e5422db1d622af5218dd37 (127.23.88.67:39285), ea21b9d3aafd43018a38584aafceecb8 (127.23.88.65:45683)
I20260805 23:19:31.811492 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.808079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:42281 (local address 127.23.88.65:45683)
0805 23:19:31.808223 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.808227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.808240 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:19:31.808308 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.808312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.808368 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.808453 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:31.808790 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.809273 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.810468 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.810850 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.810988 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.811043 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.811269 (+ 226us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.811350 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:19:31.811492 11617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.807656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:39285 (local address 127.23.88.66:38877)
0805 23:19:31.807872 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.807889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.807986 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.808390 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.808396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.808419 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:31.808553 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.808568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.809443 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.809447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.810219 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.810227 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.811023 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.811027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.811186 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.811203 (+ 17us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.811298 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.811362 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":1}
I20260805 23:19:31.811558 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.807765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:38877 (local address 127.23.88.67:39285)
0805 23:19:31.807901 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.807908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.807953 (+ 45us) server_negotiation.cc:407] Connection header received
0805 23:19:31.808157 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.808161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.808217 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.808358 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:31.808685 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.809296 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.810362 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.810851 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.810998 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.811049 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.811347 (+ 298us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.811436 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:19:31.811513 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.807715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45683 (local address 127.23.88.66:42281)
0805 23:19:31.808035 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.808076 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:31.808180 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:31.808495 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:31.808498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:31.808525 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:31.808665 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.808674 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.809399 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.809406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.810326 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:31.810342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.810987 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.810991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:31.811131 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.811161 (+ 30us) client_negotiation.cc:769] Sending connection context
0805 23:19:31.811229 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 23:19:31.811333 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":81,"threads_started":1}
I20260805 23:19:31.812273 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991936956f044b558ef5fa056b9e89fa" candidate_uuid: "c9221f2f93f54aae92f8ec01268edd78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7fd8acc1e5422db1d622af5218dd37" is_pre_election: true
I20260805 23:19:31.812592 11710 raft_consensus.cc:2454] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9221f2f93f54aae92f8ec01268edd78 in term 0.
I20260805 23:19:31.812565 11423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991936956f044b558ef5fa056b9e89fa" candidate_uuid: "c9221f2f93f54aae92f8ec01268edd78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea21b9d3aafd43018a38584aafceecb8" is_pre_election: true
I20260805 23:19:31.812885 11423 raft_consensus.cc:2454] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9221f2f93f54aae92f8ec01268edd78 in term 0.
I20260805 23:19:31.813195 11497 leader_election.cc:302] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f7fd8acc1e5422db1d622af5218dd37, c9221f2f93f54aae92f8ec01268edd78; no voters:
I20260805 23:19:31.813532 11790 raft_consensus.cc:2781] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:31.813619 11790 raft_consensus.cc:482] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:31.813668 11790 raft_consensus.cc:3037] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:31.814700 11790 raft_consensus.cc:504] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.815068 11790 leader_election.cc:288] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [CANDIDATE]: Term 1 election: Requested vote from peers 9f7fd8acc1e5422db1d622af5218dd37 (127.23.88.67:39285), ea21b9d3aafd43018a38584aafceecb8 (127.23.88.65:45683)
I20260805 23:19:31.815412 11710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991936956f044b558ef5fa056b9e89fa" candidate_uuid: "c9221f2f93f54aae92f8ec01268edd78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f7fd8acc1e5422db1d622af5218dd37"
I20260805 23:19:31.815527 11423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "991936956f044b558ef5fa056b9e89fa" candidate_uuid: "c9221f2f93f54aae92f8ec01268edd78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea21b9d3aafd43018a38584aafceecb8"
I20260805 23:19:31.815549 11710 raft_consensus.cc:3037] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:31.815682 11423 raft_consensus.cc:3037] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:31.816799 11710 raft_consensus.cc:2454] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9221f2f93f54aae92f8ec01268edd78 in term 1.
I20260805 23:19:31.817055 11423 raft_consensus.cc:2454] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9221f2f93f54aae92f8ec01268edd78 in term 1.
I20260805 23:19:31.817095 11497 leader_election.cc:302] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f7fd8acc1e5422db1d622af5218dd37, c9221f2f93f54aae92f8ec01268edd78; no voters:
I20260805 23:19:31.817304 11790 raft_consensus.cc:2781] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:31.817562 11790 raft_consensus.cc:686] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 1 LEADER]: Becoming Leader. State: Replica: c9221f2f93f54aae92f8ec01268edd78, State: Running, Role: LEADER
I20260805 23:19:31.818027 11790 consensus_queue.cc:237] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } }
I20260805 23:19:31.821378 11267 catalog_manager.cc:5495] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9221f2f93f54aae92f8ec01268edd78 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "c9221f2f93f54aae92f8ec01268edd78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9221f2f93f54aae92f8ec01268edd78" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 42677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 } health_report { overall_health: UNKNOWN } } }
I20260805 23:19:31.840879 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.836667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.23.88.66:42677)
0805 23:19:31.836840 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:31.836845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:31.836864 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:19:31.837007 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:31.837011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:31.837085 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:31.837216 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:31.837506 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.838318 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.839309 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:31.839597 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:31.839714 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:31.840043 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:31.840420 (+ 377us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:31.840586 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:31.840676 (+ 90us) server_negotiation.cc:299] Negotiation successful
0805 23:19:31.840716 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260805 23:19:31.847957 11423 raft_consensus.cc:1261] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 1 FOLLOWER]: Refusing update from remote peer c9221f2f93f54aae92f8ec01268edd78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:31.847957 11710 raft_consensus.cc:1261] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 1 FOLLOWER]: Refusing update from remote peer c9221f2f93f54aae92f8ec01268edd78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:31.848872 11792 consensus_queue.cc:1035] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f7fd8acc1e5422db1d622af5218dd37" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 39285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:31.849105 11790 consensus_queue.cc:1035] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea21b9d3aafd43018a38584aafceecb8" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:31.859668 11797 mvcc.cc:205] Tried to move back new op lower bound from 7315341196680630272 to 7315341196569214976. Current Snapshot: MvccSnapshot[applied={T|T < 7315341196680630272}]
I20260805 23:19:31.867844 11268 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:31.868021 11268 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:31.870942 11268 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e133832ee3d44d46a672f32dfdd4be7f: Sending DeleteTablet for 3 replicas of tablet 991936956f044b558ef5fa056b9e89fa
I20260805 23:19:31.871878 11401 tablet_service.cc:1514] Processing DeleteTablet for tablet 991936956f044b558ef5fa056b9e89fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:31 UTC) from {username='slave'} at 127.0.0.1:50156
I20260805 23:19:31.871968 11544 tablet_service.cc:1514] Processing DeleteTablet for tablet 991936956f044b558ef5fa056b9e89fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:31 UTC) from {username='slave'} at 127.0.0.1:58156
I20260805 23:19:31.872144 11690 tablet_service.cc:1514] Processing DeleteTablet for tablet 991936956f044b558ef5fa056b9e89fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:31 UTC) from {username='slave'} at 127.0.0.1:37924
I20260805 23:19:31.872491 11806 tablet_replica.cc:330] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8: stopping tablet replica
I20260805 23:19:31.872489 11807 tablet_replica.cc:330] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: stopping tablet replica
I20260805 23:19:31.872859 11806 raft_consensus.cc:2229] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:31.872996 11808 tablet_replica.cc:330] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: stopping tablet replica
I20260805 23:19:31.873087 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11335
I20260805 23:19:31.872917 11807 raft_consensus.cc:2229] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:19:31.873222 11806 raft_consensus.cc:2258] T 991936956f044b558ef5fa056b9e89fa P ea21b9d3aafd43018a38584aafceecb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:31.873270 11808 raft_consensus.cc:2229] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:31.873538 11808 raft_consensus.cc:2258] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:31.873749 11807 raft_consensus.cc:2258] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:31.875633 11807 ts_tablet_manager.cc:1905] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:31.876593 11808 ts_tablet_manager.cc:1905] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:31.879421 11808 ts_tablet_manager.cc:1918] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:31.879427 11807 ts_tablet_manager.cc:1918] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:31.879530 11808 log.cc:1196] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/991936956f044b558ef5fa056b9e89fa
I20260805 23:19:31.879679 11807 log.cc:1196] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/991936956f044b558ef5fa056b9e89fa
I20260805 23:19:31.879884 11808 ts_tablet_manager.cc:1939] T 991936956f044b558ef5fa056b9e89fa P 9f7fd8acc1e5422db1d622af5218dd37: Deleting consensus metadata
I20260805 23:19:31.880173 11807 ts_tablet_manager.cc:1939] T 991936956f044b558ef5fa056b9e89fa P c9221f2f93f54aae92f8ec01268edd78: Deleting consensus metadata
I20260805 23:19:31.881348 11255 catalog_manager.cc:4841] TS 9f7fd8acc1e5422db1d622af5218dd37 (127.23.88.67:39285): tablet 991936956f044b558ef5fa056b9e89fa (table test-table [id=253c60ababde4790be055abd3ef25d77]) successfully deleted
I20260805 23:19:31.881599 11255 catalog_manager.cc:4841] TS c9221f2f93f54aae92f8ec01268edd78 (127.23.88.66:42677): tablet 991936956f044b558ef5fa056b9e89fa (table test-table [id=253c60ababde4790be055abd3ef25d77]) successfully deleted
W20260805 23:19:31.882557 11254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:45683 (error 108)
I20260805 23:19:31.883518 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11478
I20260805 23:19:31.883744 11777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:31.883394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45683 (local address 127.0.0.1:50168)
0805 23:19:31.883554 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:31.883636 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260805 23:19:31.888345 11254 catalog_manager.cc:4584] TS ea21b9d3aafd43018a38584aafceecb8 (127.23.88.65:45683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 991936956f044b558ef5fa056b9e89fa: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45683: connect: Connection refused (error 111)
I20260805 23:19:31.893033 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11618
I20260805 23:19:31.900189 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11237
2026-08-05T23:19:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1271 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:31 dist-test-slave-vkjp krb5kdc[11813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:31 dist-test-slave-vkjp krb5kdc[11813](info): set up 2 sockets
Aug 05 23:19:31 dist-test-slave-vkjp krb5kdc[11813](info): commencing operation
krb5kdc: starting...
W20260805 23:19:32.997440 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:33.131744 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42959
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:42959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:33.244728 11828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:33.245066 11828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:33.245149 11828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:33.249279 11828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:33.249372 11828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:33.249423 11828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:33.249487 11828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:33.249526 11828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:33.255002 11828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:42959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:42959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:33.256199 11828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:33.257297 11828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:33.266306 11834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:33.266322 11833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:33.266480 11828 server_base.cc:1034] running on GCE node
W20260805 23:19:33.266322 11836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:33.267424 11828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:33.268649 11828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:33.269876 11828 hybrid_clock.cc:648] HybridClock initialized: now 1785971973269867 us; error 48 us; skew 500 ppm
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:33.273000 11828 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:33.273921 11828 webserver.cc:460] Webserver started at http://127.23.88.126:41189/ using document root <none> and password file <none>
I20260805 23:19:33.274510 11828 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:33.274602 11828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:33.274844 11828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:33.277056 11828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "32edad0df0c346a7aa38e8683cfeed15"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "6b7d5541b6e91039c1e5a2e9f12255ba"
server_key_iv: "e1c38708c5698327dca57672f36c4635"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.277661 11828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "32edad0df0c346a7aa38e8683cfeed15"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "6b7d5541b6e91039c1e5a2e9f12255ba"
server_key_iv: "e1c38708c5698327dca57672f36c4635"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.281613 11828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:33.284226 11844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.285573 11828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:33.285719 11828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "32edad0df0c346a7aa38e8683cfeed15"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "6b7d5541b6e91039c1e5a2e9f12255ba"
server_key_iv: "e1c38708c5698327dca57672f36c4635"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.285841 11828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:33.309870 11828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:33.313428 11828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:33.313673 11828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:33.321667 11828 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:42959
I20260805 23:19:33.321692 11900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:42959 every 8 connection(s)
I20260805 23:19:33.322705 11828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:33.325614 11901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:33.328801 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11828
I20260805 23:19:33.328922 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:33.329226 23905 external_mini_cluster.cc:1428] Setting key 41577f6b9cc33a13ebcf88c3db087f90
I20260805 23:19:33.331833 11901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: Bootstrap starting.
I20260805 23:19:33.334463 11901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:33.335453 11901 log.cc:824] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:33.337863 11901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: No bootstrap required, opened a new log
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:33.341531 11901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } }
I20260805 23:19:33.341791 11901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:33.341874 11901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32edad0df0c346a7aa38e8683cfeed15, State: Initialized, Role: FOLLOWER
I20260805 23:19:33.342370 11901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } }
I20260805 23:19:33.342526 11901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:33.342615 11901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:33.342749 11901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:33.343765 11901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } }
I20260805 23:19:33.344147 11901 leader_election.cc:302] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32edad0df0c346a7aa38e8683cfeed15; no voters:
I20260805 23:19:33.344470 11901 leader_election.cc:288] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:33.344605 11906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:33.344810 11906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [term 1 LEADER]: Becoming Leader. State: Replica: 32edad0df0c346a7aa38e8683cfeed15, State: Running, Role: LEADER
I20260805 23:19:33.345007 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:33.330766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.23.88.126:42959)
0805 23:19:33.331224 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:33.331241 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:33.331276 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:19:33.331915 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:33.331932 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:33.332307 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:33.332581 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:33.333549 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.334532 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.335751 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.336415 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.336536 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:33.339364 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:33.339369 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:33.339382 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:33.339410 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:33.341653 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.342159 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.342164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.342168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.342279 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.342597 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.342600 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.342602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.343194 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:33.343409 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:33.343689 (+ 280us) server_negotiation.cc:299] Negotiation successful
0805 23:19:33.344013 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":156,"threads_started":1}
I20260805 23:19:33.345144 11906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } }
I20260805 23:19:33.345855 11901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:33.346326 11907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32edad0df0c346a7aa38e8683cfeed15. Latest consensus state: current_term: 1 leader_uuid: "32edad0df0c346a7aa38e8683cfeed15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } } }
I20260805 23:19:33.346467 11907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:33.346767 11908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32edad0df0c346a7aa38e8683cfeed15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edad0df0c346a7aa38e8683cfeed15" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 42959 } } }
I20260805 23:19:33.346843 11908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15 [sys.catalog]: This master's current role is: LEADER
W20260805 23:19:33.350438 11922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:19:33.350507 11922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:19:33.350596 11911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:33.351540 11911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:33.357321 11911 catalog_manager.cc:1347] Generated new cluster ID: 469d80b309d7475bb520c0babd37fdb7
I20260805 23:19:33.357402 11911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:33.365594 11911 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:33.366569 11911 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:33.372371 11911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: Generated new TSK 0
I20260805 23:19:33.373235 11911 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.88.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
Entry for principal kudu/127.23.88.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab.
I20260805 23:19:33.408654 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:33.524694 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:33.525012 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:33.525087 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:33.529148 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:33.529268 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:33.529376 11928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:33.534837 11928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.65.keytab
--principal=kudu/127.23.88.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:33.536000 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:33.537052 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:33.546805 11934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:33.546854 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:33.546983 11928 server_base.cc:1034] running on GCE node
W20260805 23:19:33.546860 11933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:33.547632 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:33.548328 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:33.549537 11928 hybrid_clock.cc:648] HybridClock initialized: now 1785971973549505 us; error 52 us; skew 500 ppm
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:33.552722 11928 init.cc:377] Logged in from keytab as kudu/127.23.88.65@KRBTEST.COM (short username kudu)
I20260805 23:19:33.553699 11928 webserver.cc:460] Webserver started at http://127.23.88.65:38161/ using document root <none> and password file <none>
I20260805 23:19:33.554288 11928 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:33.554363 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:33.554548 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:33.556937 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "48c1da895e8545d8a381aaf7e1aadd5b"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "fd3a29fb5ebb5bd0680da1b36db97b89"
server_key_iv: "363d5635e7d5fafe5a8158bc7ebc138b"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.557528 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "48c1da895e8545d8a381aaf7e1aadd5b"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "fd3a29fb5ebb5bd0680da1b36db97b89"
server_key_iv: "363d5635e7d5fafe5a8158bc7ebc138b"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.561800 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260805 23:19:33.564695 11943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.566242 11928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260805 23:19:33.566349 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "48c1da895e8545d8a381aaf7e1aadd5b"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "fd3a29fb5ebb5bd0680da1b36db97b89"
server_key_iv: "363d5635e7d5fafe5a8158bc7ebc138b"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.566448 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:33.595817 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:33.599511 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:33.599751 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:33.600437 11928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:33.601397 11928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:33.601468 11928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.601538 11928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:33.601565 11928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.610862 11928 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:44293
I20260805 23:19:33.610891 12063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:44293 every 8 connection(s)
I20260805 23:19:33.611847 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:33.616494 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 11928
I20260805 23:19:33.616611 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:33.616909 23905 external_mini_cluster.cc:1428] Setting key d71003d1749171fa42278b99479351a3
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.65@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.66@KRBTEST.COM; defaulting to no policy
I20260805 23:19:33.628183 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:33.613888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:52457 (local address 127.23.88.126:42959)
0805 23:19:33.614109 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:33.614113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:33.615153 (+ 1040us) server_negotiation.cc:407] Connection header received
0805 23:19:33.616123 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:33.616126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:33.616201 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:33.616298 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:33.617884 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.618443 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.619696 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.620128 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.620251 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:33.623574 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:33.623581 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:33.623585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:33.623628 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:33.625655 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.626261 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.626268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.626272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.626376 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.626746 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.626749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.626751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.627038 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:33.627204 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:33.627824 (+ 620us) server_negotiation.cc:299] Negotiation successful
0805 23:19:33.627992 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260805 23:19:33.629076 12066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:33.614228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42959 (local address 127.23.88.65:52457)
0805 23:19:33.614946 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:33.614991 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:33.615927 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:33.616502 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:33.616509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:33.616979 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:33.617740 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:33.617762 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.618577 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.618579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:33.619568 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:33.619579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.620338 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.620345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:33.620653 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:33.621361 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:33.621388 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:33.623349 (+ 1961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:33.625801 (+ 2452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:33.625807 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:33.625825 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:33.626127 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:33.626478 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:33.626480 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:33.626482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:33.626631 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:33.627226 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:33.627232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:33.627530 (+ 298us) client_negotiation.cc:769] Sending connection context
0805 23:19:33.627708 (+ 178us) client_negotiation.cc:240] Negotiation successful
0805 23:19:33.628058 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":137,"threads_started":1}
I20260805 23:19:33.630496 12064 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42959
I20260805 23:19:33.630757 12064 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.66@KRBTEST.COM" created.
I20260805 23:19:33.631366 12064 heartbeater.cc:511] Master 127.23.88.126:42959 requested a full tablet report, sending...
I20260805 23:19:33.633112 11861 ts_manager.cc:194] Registered new tserver with Master: 48c1da895e8545d8a381aaf7e1aadd5b (127.23.88.65:44293)
I20260805 23:19:33.634943 11861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.65@KRBTEST.COM'} at 127.23.88.65:52457
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
Entry for principal kudu/127.23.88.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab.
I20260805 23:19:33.650595 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:33.768321 12069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:33.768599 12069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:33.768668 12069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:33.772936 12069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:33.773005 12069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:33.773098 12069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:33.778708 12069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.66.keytab
--principal=kudu/127.23.88.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:33.779908 12069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:33.781052 12069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:33.790867 12075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:33.790894 12074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:33.791339 12077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:33.791307 12069 server_base.cc:1034] running on GCE node
I20260805 23:19:33.791797 12069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:33.792450 12069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:33.793639 12069 hybrid_clock.cc:648] HybridClock initialized: now 1785971973793621 us; error 28 us; skew 500 ppm
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:33.796712 12069 init.cc:377] Logged in from keytab as kudu/127.23.88.66@KRBTEST.COM (short username kudu)
I20260805 23:19:33.797698 12069 webserver.cc:460] Webserver started at http://127.23.88.66:38983/ using document root <none> and password file <none>
I20260805 23:19:33.798308 12069 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:33.798383 12069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:33.798640 12069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:33.800935 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "a84a678bdb70bbdb578aa5af0f4568d9"
server_key_iv: "05a4a6101488352001f9b45d73bc3c04"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.801584 12069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "a84a678bdb70bbdb578aa5af0f4568d9"
server_key_iv: "05a4a6101488352001f9b45d73bc3c04"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.806243 12069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260805 23:19:33.808980 12084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.810209 12069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:33.810348 12069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617"
format_stamp: "Formatted at 2026-08-05 23:19:33 on dist-test-slave-vkjp"
server_key: "a84a678bdb70bbdb578aa5af0f4568d9"
server_key_iv: "05a4a6101488352001f9b45d73bc3c04"
server_key_version: "encryptionkey@0"
I20260805 23:19:33.810473 12069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:33.831923 12069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:33.835186 12069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:33.835377 12069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:33.836061 12069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:33.837041 12069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:33.837090 12069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.837162 12069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:33.837195 12069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:33.846774 12069 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:38993
I20260805 23:19:33.846815 12203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:38993 every 8 connection(s)
I20260805 23:19:33.847883 12069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
Aug 05 23:19:33 dist-test-slave-vkjp krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971973, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.66@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:33.857921 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12069
I20260805 23:19:33.858052 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:33.858376 23905 external_mini_cluster.cc:1428] Setting key 82604da1f15a91f17da08f85256f42f3
I20260805 23:19:33.862133 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:33.849852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:46079 (local address 127.23.88.126:42959)
0805 23:19:33.850011 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:33.850016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:33.851140 (+ 1124us) server_negotiation.cc:407] Connection header received
0805 23:19:33.852104 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:33.852107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:33.852155 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:33.852234 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:33.853454 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.854001 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.855212 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.855425 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.855516 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:33.858327 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:33.858332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:33.858336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:33.858367 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:33.859914 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.860419 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.860425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.860430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.860521 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:33.860844 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:33.860850 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:33.860855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:33.861137 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:33.861241 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:33.861752 (+ 511us) server_negotiation.cc:299] Negotiation successful
0805 23:19:33.861899 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260805 23:19:33.862906 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:33.850274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42959 (local address 127.23.88.66:46079)
0805 23:19:33.850960 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:33.851005 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:33.851908 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:33.852431 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:33.852437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:33.852800 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:33.853288 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:33.853301 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.854115 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.854118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:33.855014 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:33.855024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:33.855598 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:33.855602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:33.855765 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:33.856502 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:33.856520 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:33.858134 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:33.860045 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:33.860050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:33.860062 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:33.860316 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:33.860613 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:33.860615 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:33.860617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:33.860754 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:33.861240 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:33.861245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:33.861477 (+ 232us) client_negotiation.cc:769] Sending connection context
0805 23:19:33.861672 (+ 195us) client_negotiation.cc:240] Negotiation successful
0805 23:19:33.861940 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":122,"threads_started":1}
I20260805 23:19:33.864133 12204 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42959
I20260805 23:19:33.864391 12204 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:33.864881 12204 heartbeater.cc:511] Master 127.23.88.126:42959 requested a full tablet report, sending...
I20260805 23:19:33.865998 11861 ts_manager.cc:194] Registered new tserver with Master: cf2a91d2f5aa4dd1aa04f449e16a9617 (127.23.88.66:38993)
I20260805 23:19:33.866909 11861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.66@KRBTEST.COM'} at 127.23.88.66:46079
WARNING: no policy specified for kudu/127.23.88.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
Entry for principal kudu/127.23.88.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab.
I20260805 23:19:33.891144 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:37101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:34.005261 12209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:34.005540 12209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:34.005586 12209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:34.009922 12209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:34.010007 12209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:34.010068 12209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:34.015866 12209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:37101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.67.keytab
--principal=kudu/127.23.88.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:42959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:34.017112 12209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:34.018306 12209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:34.028664 12215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.028651 12217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.028656 12214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.028920 12209 server_base.cc:1034] running on GCE node
I20260805 23:19:34.029462 12209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:34.030268 12209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:34.031468 12209 hybrid_clock.cc:648] HybridClock initialized: now 1785971974031446 us; error 31 us; skew 500 ppm
Aug 05 23:19:34 dist-test-slave-vkjp krb5kdc[11813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971974, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:34.034353 12209 init.cc:377] Logged in from keytab as kudu/127.23.88.67@KRBTEST.COM (short username kudu)
I20260805 23:19:34.035364 12209 webserver.cc:460] Webserver started at http://127.23.88.67:37247/ using document root <none> and password file <none>
I20260805 23:19:34.035949 12209 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:34.036044 12209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:34.036283 12209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:34.038440 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "48fcd61ba4c44d0baef97826ea207bed"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "5f53b69d6eeb0d25d077f3533f4759a6"
server_key_iv: "184e153ad5bfdff1cd4a28212854d509"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.039029 12209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "48fcd61ba4c44d0baef97826ea207bed"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "5f53b69d6eeb0d25d077f3533f4759a6"
server_key_iv: "184e153ad5bfdff1cd4a28212854d509"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.042934 12209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260805 23:19:34.045516 12224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.046921 12209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:19:34.047072 12209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "48fcd61ba4c44d0baef97826ea207bed"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "5f53b69d6eeb0d25d077f3533f4759a6"
server_key_iv: "184e153ad5bfdff1cd4a28212854d509"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.047271 12209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:34.061190 12209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:34.064603 12209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:34.064813 12209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:34.065413 12209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:34.066506 12209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:34.066597 12209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.066677 12209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:34.066725 12209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.075923 12209 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:34975
I20260805 23:19:34.075986 12347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:34975 every 8 connection(s)
I20260805 23:19:34.076894 12209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:34.078215 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12209
I20260805 23:19:34.078361 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:34.078720 23905 external_mini_cluster.cc:1428] Setting key 75799cb744c1270ffa5dd979156d738c
Aug 05 23:19:34 dist-test-slave-vkjp krb5kdc[11813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971974, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.67@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:34.091506 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.078995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:53325 (local address 127.23.88.126:42959)
0805 23:19:34.079422 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.079426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.080686 (+ 1260us) server_negotiation.cc:407] Connection header received
0805 23:19:34.081742 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.081745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.081814 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.081900 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:34.083048 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.083722 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.084831 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.085066 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.085170 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.087947 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:34.087950 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:34.087952 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:34.087978 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:34.089507 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:34.090010 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:34.090013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:34.090014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:34.090098 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:34.090444 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:34.090447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:34.090448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:34.090638 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:34.090725 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.091240 (+ 515us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.091367 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
I20260805 23:19:34.092568 12351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.079665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:42959 (local address 127.23.88.67:53325)
0805 23:19:34.080510 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.080568 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.081529 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.082043 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.082050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.082399 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:34.082897 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.082910 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.083845 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.083847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.084705 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.084714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.085213 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.085216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.085350 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.085995 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:19:34.086011 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:34.087776 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:34.089633 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:34.089641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:34.089658 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:34.089918 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:34.090181 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:19:34.090183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:19:34.090184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:19:34.090358 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:19:34.090732 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:34.090737 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:34.090967 (+ 230us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.091187 (+ 220us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.091526 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":157,"threads_started":1}
I20260805 23:19:34.093716 12348 heartbeater.cc:347] Connected to a master server at 127.23.88.126:42959
I20260805 23:19:34.093927 12348 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:34.094365 12348 heartbeater.cc:511] Master 127.23.88.126:42959 requested a full tablet report, sending...
I20260805 23:19:34.095366 11861 ts_manager.cc:194] Registered new tserver with Master: 48fcd61ba4c44d0baef97826ea207bed (127.23.88.67:34975)
I20260805 23:19:34.095934 11861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.88.67@KRBTEST.COM'} at 127.23.88.67:53325
I20260805 23:19:34.104884 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:34.114629 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.106867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60592 (local address 127.23.88.126:42959)
0805 23:19:34.106974 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.106978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.107283 (+ 305us) server_negotiation.cc:407] Connection header received
0805 23:19:34.107449 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.107453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.107505 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.107587 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:34.108303 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.108829 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.109683 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.109907 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.110011 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.111485 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:34.111488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:34.111490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:34.111513 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:34.112783 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:34.113214 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:34.113217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:34.113219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:34.113303 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:34.113614 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:34.113618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:34.113620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:34.113851 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:34.113996 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.114279 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.114400 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 23:19:34.117834 11861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:34.121126 11861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:34.136843 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.130903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50262 (local address 127.23.88.65:44293)
0805 23:19:34.132336 (+ 1433us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.132341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.132359 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:19:34.132420 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.132424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.132602 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.133005 (+ 403us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:34.133145 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.134251 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.135546 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.136110 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.136237 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.136393 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.136576 (+ 183us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.136672 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":86,"threads_started":1}
I20260805 23:19:34.136828 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.130722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44293 (local address 127.0.0.1:50262)
0805 23:19:34.131554 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.131609 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.131740 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.132740 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.132743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.132757 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:34.132827 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.132834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.134382 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.134385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.135422 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.135431 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.136238 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.136242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.136362 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.136413 (+ 51us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.136542 (+ 129us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.136668 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":51,"thread_start_us":105,"threads_started":1}
I20260805 23:19:34.137146 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.131461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.23.88.66:38993)
0805 23:19:34.131936 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.131962 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.132133 (+ 171us) server_negotiation.cc:407] Connection header received
0805 23:19:34.132223 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.132229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.132432 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.132597 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:34.132925 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.133712 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.135353 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.136210 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.136310 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.136439 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.136897 (+ 458us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.137006 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":156,"threads_started":1}
I20260805 23:19:34.137632 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.130913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38993 (local address 127.0.0.1:46428)
0805 23:19:34.132015 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.132029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.132122 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.132602 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.132608 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.132630 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:34.132804 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.132811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.133841 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.133846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.135229 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.135247 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.136503 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.136509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.136600 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.136623 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.136687 (+ 64us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.136756 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"mutex_wait_us":54,"spinlock_wait_cycles":7424,"thread_start_us":105,"threads_started":1}
I20260805 23:19:34.138159 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.131200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:34975 (local address 127.0.0.1:56032)
0805 23:19:34.132015 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.132029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.132122 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.132782 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.132784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.132800 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:34.132912 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.132918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.134386 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.134388 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.136451 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.136464 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.137750 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.137753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.137849 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.137872 (+ 23us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.137940 (+ 68us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.138011 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":34,"spinlock_wait_cycles":3584,"thread_start_us":90,"threads_started":1}
I20260805 23:19:34.138543 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.131650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.23.88.67:34975)
0805 23:19:34.132232 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.132243 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.132262 (+ 19us) server_negotiation.cc:407] Connection header received
0805 23:19:34.132325 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.132329 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.132483 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.132607 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:34.133039 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.134218 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.136591 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.137408 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.137544 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.137691 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.138327 (+ 636us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.138427 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":150,"threads_started":1}
I20260805 23:19:34.140463 11995 tablet_service.cc:1467] Processing CreateTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 (DEFAULT_TABLE table=test-table [id=2eb89a73fd834039bb035200cf730c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:34.140465 12135 tablet_service.cc:1467] Processing CreateTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 (DEFAULT_TABLE table=test-table [id=2eb89a73fd834039bb035200cf730c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:34.140765 12278 tablet_service.cc:1467] Processing CreateTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 (DEFAULT_TABLE table=test-table [id=2eb89a73fd834039bb035200cf730c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:34.141637 11995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66983da4324df1af0c0f8d5a47ab69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:34.141636 12135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66983da4324df1af0c0f8d5a47ab69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:34.141762 12278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee66983da4324df1af0c0f8d5a47ab69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:34.148209 12366 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Bootstrap starting.
I20260805 23:19:34.149718 12368 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Bootstrap starting.
I20260805 23:19:34.150827 12366 tablet_bootstrap.cc:654] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:34.151111 12367 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: Bootstrap starting.
I20260805 23:19:34.152606 12366 log.cc:824] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:34.152721 12368 tablet_bootstrap.cc:654] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:34.153292 12367 tablet_bootstrap.cc:654] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:34.153585 12368 log.cc:824] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:34.154099 12367 log.cc:824] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:34.155169 12366 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: No bootstrap required, opened a new log
I20260805 23:19:34.155376 12366 ts_tablet_manager.cc:1397] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260805 23:19:34.155687 12368 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: No bootstrap required, opened a new log
I20260805 23:19:34.155866 12368 ts_tablet_manager.cc:1397] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:34.156033 12367 tablet_bootstrap.cc:492] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: No bootstrap required, opened a new log
I20260805 23:19:34.156227 12367 ts_tablet_manager.cc:1397] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260805 23:19:34.159615 12368 raft_consensus.cc:348] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.159615 12366 raft_consensus.cc:348] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.159938 12366 raft_consensus.cc:374] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:34.159937 12368 raft_consensus.cc:374] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:34.159993 12368 raft_consensus.cc:729] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf2a91d2f5aa4dd1aa04f449e16a9617, State: Initialized, Role: FOLLOWER
I20260805 23:19:34.159993 12366 raft_consensus.cc:729] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48fcd61ba4c44d0baef97826ea207bed, State: Initialized, Role: FOLLOWER
I20260805 23:19:34.159817 12367 raft_consensus.cc:348] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.160076 12367 raft_consensus.cc:374] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:34.160136 12367 raft_consensus.cc:729] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c1da895e8545d8a381aaf7e1aadd5b, State: Initialized, Role: FOLLOWER
I20260805 23:19:34.160480 12368 consensus_queue.cc:260] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.160480 12366 consensus_queue.cc:260] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.160597 12367 consensus_queue.cc:260] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.161477 12204 heartbeater.cc:503] Master 127.23.88.126:42959 was elected leader, sending a full tablet report...
I20260805 23:19:34.161618 12367 ts_tablet_manager.cc:1428] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260805 23:19:34.161621 12366 ts_tablet_manager.cc:1428] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:19:34.161621 12368 ts_tablet_manager.cc:1428] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260805 23:19:34.161974 12064 heartbeater.cc:503] Master 127.23.88.126:42959 was elected leader, sending a full tablet report...
I20260805 23:19:34.162114 12348 heartbeater.cc:503] Master 127.23.88.126:42959 was elected leader, sending a full tablet report...
I20260805 23:19:34.171916 12374 raft_consensus.cc:482] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:34.172091 12374 raft_consensus.cc:504] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.173113 12374 leader_election.cc:288] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf2a91d2f5aa4dd1aa04f449e16a9617 (127.23.88.66:38993), 48fcd61ba4c44d0baef97826ea207bed (127.23.88.67:34975)
I20260805 23:19:34.177208 12365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.173612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:42711 (local address 127.23.88.67:34975)
0805 23:19:34.173738 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.173743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.173924 (+ 181us) server_negotiation.cc:407] Connection header received
0805 23:19:34.174087 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.174091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.174138 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.174217 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:34.174524 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.175033 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176205 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.176614 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176744 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.176796 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.176985 (+ 189us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.177077 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:19:34.177207 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.173461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:52169 (local address 127.23.88.66:38993)
0805 23:19:34.173627 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.173634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.173983 (+ 349us) server_negotiation.cc:407] Connection header received
0805 23:19:34.174129 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.174138 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.174180 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.174264 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:34.174559 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.175061 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176217 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.176639 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176766 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.176822 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.176999 (+ 177us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.177070 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260805 23:19:34.177237 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.173394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38993 (local address 127.23.88.65:52169)
0805 23:19:34.173907 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.173920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.174009 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.174275 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.174278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.174315 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:34.174428 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.174433 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.175240 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.175246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.176096 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.176105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.176762 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.176876 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.176894 (+ 18us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.176982 (+ 88us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.177070 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":143,"spinlock_wait_cycles":1280,"thread_start_us":78,"threads_started":1}
I20260805 23:19:34.177248 12376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.173394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:34975 (local address 127.23.88.65:42711)
0805 23:19:34.173827 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.173840 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.173985 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.174227 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.174230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.174245 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:34.174411 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.174417 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.175170 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.175176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.176080 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.176090 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.176741 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.176745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.176840 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.176867 (+ 27us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.176959 (+ 92us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.177077 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":64,"spinlock_wait_cycles":31744,"thread_start_us":137,"threads_started":1}
I20260805 23:19:34.178031 12300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66983da4324df1af0c0f8d5a47ab69" candidate_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fcd61ba4c44d0baef97826ea207bed" is_pre_election: true
I20260805 23:19:34.178016 12158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66983da4324df1af0c0f8d5a47ab69" candidate_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" is_pre_election: true
I20260805 23:19:34.178366 12158 raft_consensus.cc:2454] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c1da895e8545d8a381aaf7e1aadd5b in term 0.
I20260805 23:19:34.178366 12300 raft_consensus.cc:2454] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48c1da895e8545d8a381aaf7e1aadd5b in term 0.
I20260805 23:19:34.178931 11945 leader_election.cc:302] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 48c1da895e8545d8a381aaf7e1aadd5b, 48fcd61ba4c44d0baef97826ea207bed, cf2a91d2f5aa4dd1aa04f449e16a9617; no voters:
I20260805 23:19:34.179289 12374 raft_consensus.cc:2781] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:34.179374 12374 raft_consensus.cc:482] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:34.179407 12374 raft_consensus.cc:3037] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:34.180501 12374 raft_consensus.cc:504] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.180852 12374 leader_election.cc:288] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [CANDIDATE]: Term 1 election: Requested vote from peers cf2a91d2f5aa4dd1aa04f449e16a9617 (127.23.88.66:38993), 48fcd61ba4c44d0baef97826ea207bed (127.23.88.67:34975)
I20260805 23:19:34.181250 12158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66983da4324df1af0c0f8d5a47ab69" candidate_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617"
I20260805 23:19:34.181250 12300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee66983da4324df1af0c0f8d5a47ab69" candidate_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48fcd61ba4c44d0baef97826ea207bed"
I20260805 23:19:34.181391 12158 raft_consensus.cc:3037] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:34.181391 12300 raft_consensus.cc:3037] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:34.182760 12300 raft_consensus.cc:2454] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c1da895e8545d8a381aaf7e1aadd5b in term 1.
I20260805 23:19:34.182760 12158 raft_consensus.cc:2454] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48c1da895e8545d8a381aaf7e1aadd5b in term 1.
I20260805 23:19:34.183133 11945 leader_election.cc:302] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 48c1da895e8545d8a381aaf7e1aadd5b, 48fcd61ba4c44d0baef97826ea207bed, cf2a91d2f5aa4dd1aa04f449e16a9617; no voters:
I20260805 23:19:34.183291 12374 raft_consensus.cc:2781] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:34.183519 12374 raft_consensus.cc:686] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [term 1 LEADER]: Becoming Leader. State: Replica: 48c1da895e8545d8a381aaf7e1aadd5b, State: Running, Role: LEADER
I20260805 23:19:34.183864 12374 consensus_queue.cc:237] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } }
I20260805 23:19:34.187366 11861 catalog_manager.cc:5495] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 48c1da895e8545d8a381aaf7e1aadd5b (127.23.88.65). New cstate: current_term: 1 leader_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c1da895e8545d8a381aaf7e1aadd5b" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 44293 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 } health_report { overall_health: UNKNOWN } } }
I20260805 23:19:34.199712 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.196324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.23.88.65:44293)
0805 23:19:34.196458 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.196468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.196483 (+ 15us) server_negotiation.cc:407] Connection header received
0805 23:19:34.196556 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.196558 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.196611 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.196700 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:34.196993 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.197490 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.198337 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.198562 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.198642 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.198919 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:34.199383 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:34.199477 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.199546 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.199598 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:19:34.205796 12158 raft_consensus.cc:1261] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 1 FOLLOWER]: Refusing update from remote peer 48c1da895e8545d8a381aaf7e1aadd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:34.205796 12300 raft_consensus.cc:1261] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 1 FOLLOWER]: Refusing update from remote peer 48c1da895e8545d8a381aaf7e1aadd5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:34.206666 12377 consensus_queue.cc:1035] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf2a91d2f5aa4dd1aa04f449e16a9617" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 38993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:34.206859 12374 consensus_queue.cc:1035] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "48fcd61ba4c44d0baef97826ea207bed" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 34975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:34.216735 12383 mvcc.cc:205] Tried to move back new op lower bound from 7315341206339203072 to 7315341206259998720. Current Snapshot: MvccSnapshot[applied={T|T < 7315341206339203072}]
I20260805 23:19:34.216962 12382 mvcc.cc:205] Tried to move back new op lower bound from 7315341206339203072 to 7315341206259998720. Current Snapshot: MvccSnapshot[applied={T|T < 7315341206339203072}]
I20260805 23:19:34.223486 11860 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:34.223682 11860 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:34.226395 11860 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32edad0df0c346a7aa38e8683cfeed15: Sending DeleteTablet for 3 replicas of tablet ee66983da4324df1af0c0f8d5a47ab69
I20260805 23:19:34.227248 11995 tablet_service.cc:1514] Processing DeleteTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:34 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:50262
I20260805 23:19:34.227303 12135 tablet_service.cc:1514] Processing DeleteTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:34 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:46428
I20260805 23:19:34.227849 12393 tablet_replica.cc:330] T ee66983da4324df1af0c0f8d5a47ab69 P 48c1da895e8545d8a381aaf7e1aadd5b: stopping tablet replica
I20260805 23:19:34.227953 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11928
I20260805 23:19:34.228122 12278 tablet_service.cc:1514] Processing DeleteTablet for tablet ee66983da4324df1af0c0f8d5a47ab69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:34 UTC) from {username='kudu', principal='kudu/127.23.88.126@KRBTEST.COM'} at 127.0.0.1:56032
I20260805 23:19:34.228767 12394 tablet_replica.cc:330] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: stopping tablet replica
I20260805 23:19:34.229056 12394 raft_consensus.cc:2229] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:34.229375 12394 raft_consensus.cc:2258] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:34.231194 12395 tablet_replica.cc:330] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: stopping tablet replica
I20260805 23:19:34.231489 12395 raft_consensus.cc:2229] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:34.231781 12395 raft_consensus.cc:2258] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:34.233137 12394 ts_tablet_manager.cc:1905] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:34.233258 12395 ts_tablet_manager.cc:1905] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:19:34.235944 11845 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:44293 (error 108)
I20260805 23:19:34.236086 12394 ts_tablet_manager.cc:1918] T ee66983da4324df1af0c0f8d5a47ab69 P cf2a91d2f5aa4dd1aa04f449e16a9617: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:34.236125 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12069
I20260805 23:19:34.238433 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.238052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:44293 (local address 127.0.0.1:50270)
0805 23:19:34.238249 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.238318 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:44293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260805 23:19:34.238708 11845 catalog_manager.cc:4584] TS 48c1da895e8545d8a381aaf7e1aadd5b (127.23.88.65:44293): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ee66983da4324df1af0c0f8d5a47ab69: Network error: Client connection negotiation failed: client connection to 127.23.88.65:44293: connect: Connection refused (error 111)
I20260805 23:19:34.241289 12395 ts_tablet_manager.cc:1918] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:34.241382 12395 log.cc:1196] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/ee66983da4324df1af0c0f8d5a47ab69
I20260805 23:19:34.241709 12395 ts_tablet_manager.cc:1939] T ee66983da4324df1af0c0f8d5a47ab69 P 48fcd61ba4c44d0baef97826ea207bed: Deleting consensus metadata
I20260805 23:19:34.243352 11847 catalog_manager.cc:4841] TS 48fcd61ba4c44d0baef97826ea207bed (127.23.88.67:34975): tablet ee66983da4324df1af0c0f8d5a47ab69 (table test-table [id=2eb89a73fd834039bb035200cf730c1c]) successfully deleted
I20260805 23:19:34.247637 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12209
I20260805 23:19:34.248704 12363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.247940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:38993 (local address 127.0.0.1:46438)
0805 23:19:34.248524 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.248592 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.66:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484}
I20260805 23:19:34.254262 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 11828
2026-08-05T23:19:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2354 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-05T23:19:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:34Z Disabled control of system clock
I20260805 23:19:34.286692 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43027
--webserver_interface=127.23.88.126
--webserver_port=0
--builtin_ntp_servers=127.23.88.84:32863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:43027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:34.398743 12400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:34.399053 12400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:34.399183 12400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:34.403301 12400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:34.403366 12400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:34.403383 12400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:34.403400 12400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:34.403416 12400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:34.408393 12400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:32863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:43027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:43027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:34.409526 12400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:34.410915 12400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:34.419653 12405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.419831 12400 server_base.cc:1034] running on GCE node
W20260805 23:19:34.419670 12408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.419677 12406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.420644 12400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:34.421943 12400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:34.423189 12400 hybrid_clock.cc:648] HybridClock initialized: now 1785971974423170 us; error 33 us; skew 500 ppm
I20260805 23:19:34.425335 12400 webserver.cc:460] Webserver started at http://127.23.88.126:33639/ using document root <none> and password file <none>
I20260805 23:19:34.425968 12400 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:34.426067 12400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:34.426316 12400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:34.428534 12400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "55f5ba355c234b5e8251f2d3abaf9c4e"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "dd6004333e04113b18381c5724d352d8"
server_key_iv: "b9cbe85c2f43eeea6274ada92023a098"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.429116 12400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "55f5ba355c234b5e8251f2d3abaf9c4e"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "dd6004333e04113b18381c5724d352d8"
server_key_iv: "b9cbe85c2f43eeea6274ada92023a098"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.432912 12400 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:34.435370 12415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.436617 12400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260805 23:19:34.436764 12400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "55f5ba355c234b5e8251f2d3abaf9c4e"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "dd6004333e04113b18381c5724d352d8"
server_key_iv: "b9cbe85c2f43eeea6274ada92023a098"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.436892 12400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:34.450291 12400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:34.451056 12400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:34.451333 12400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:34.458582 12400 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:43027
I20260805 23:19:34.458590 12471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:43027 every 8 connection(s)
I20260805 23:19:34.459646 12400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:34.462693 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12400
I20260805 23:19:34.462841 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:34.462895 12472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:34.463213 23905 external_mini_cluster.cc:1428] Setting key f74a2e19142e3b113212367d0ef978f2
I20260805 23:19:34.469974 12472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: Bootstrap starting.
I20260805 23:19:34.472704 12472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:34.472968 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.464679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.23.88.126:43027)
0805 23:19:34.465317 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.465332 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.465367 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:19:34.466129 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.466146 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.466535 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.466817 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:34.467246 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.468215 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.469469 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.469954 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.470114 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.471040 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:34.471046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:34.471063 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:34.471538 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:34.471597 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:34.471619 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:34.471697 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.471861 (+ 164us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.472178 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":154,"threads_started":1}
I20260805 23:19:34.473567 12472 log.cc:824] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:34.475732 12472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: No bootstrap required, opened a new log
I20260805 23:19:34.479444 12472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } }
I20260805 23:19:34.479738 12472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:34.479820 12472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f5ba355c234b5e8251f2d3abaf9c4e, State: Initialized, Role: FOLLOWER
I20260805 23:19:34.480250 12472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } }
I20260805 23:19:34.480400 12472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:34.480495 12472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:34.480595 12472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:34.481825 12472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } }
I20260805 23:19:34.482174 12472 leader_election.cc:302] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55f5ba355c234b5e8251f2d3abaf9c4e; no voters:
I20260805 23:19:34.482517 12472 leader_election.cc:288] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:34.482679 12477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:34.482921 12477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [term 1 LEADER]: Becoming Leader. State: Replica: 55f5ba355c234b5e8251f2d3abaf9c4e, State: Running, Role: LEADER
I20260805 23:19:34.483268 12477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } }
I20260805 23:19:34.483837 12472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:34.485071 12479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55f5ba355c234b5e8251f2d3abaf9c4e. Latest consensus state: current_term: 1 leader_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } } }
I20260805 23:19:34.485071 12478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55f5ba355c234b5e8251f2d3abaf9c4e" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 43027 } } }
I20260805 23:19:34.485253 12478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:34.485253 12479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:34.485754 12486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:34.489688 12486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:34.495738 12486 catalog_manager.cc:1347] Generated new cluster ID: 93afce7652214704aab0f0aa7d803f80
I20260805 23:19:34.495829 12486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:34.514354 12486 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:34.515497 12486 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:34.522747 12486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: Generated new TSK 0
I20260805 23:19:34.523548 12486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:34.533310 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.65:0
--local_ip_for_outbound_sockets=127.23.88.65
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--builtin_ntp_servers=127.23.88.84:32863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260805 23:19:34.649538 12497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:34.649801 12497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:34.649840 12497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:34.653975 12497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:34.654037 12497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:34.654129 12497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.65
I20260805 23:19:34.658921 12497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:32863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.88.65
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.65
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:34.660126 12497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:34.661582 12497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:34.671627 12503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.671693 12497 server_base.cc:1034] running on GCE node
W20260805 23:19:34.671643 12502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.671957 12505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.672399 12497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:34.673069 12497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:34.674252 12497 hybrid_clock.cc:648] HybridClock initialized: now 1785971974674241 us; error 26 us; skew 500 ppm
I20260805 23:19:34.676158 12497 webserver.cc:460] Webserver started at http://127.23.88.65:41327/ using document root <none> and password file <none>
I20260805 23:19:34.676743 12497 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:34.676838 12497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:34.677083 12497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:34.679273 12497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data/instance:
uuid: "efc9d685aa6a44fabba9cdfc5227cb78"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "9707ce54ed7ceac59bb36840909125c7"
server_key_iv: "2d2640a873858b39dc019bd47aae5db6"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.679875 12497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance:
uuid: "efc9d685aa6a44fabba9cdfc5227cb78"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "9707ce54ed7ceac59bb36840909125c7"
server_key_iv: "2d2640a873858b39dc019bd47aae5db6"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.683811 12497 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260805 23:19:34.686627 12511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.687929 12497 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260805 23:19:34.688067 12497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal
uuid: "efc9d685aa6a44fabba9cdfc5227cb78"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "9707ce54ed7ceac59bb36840909125c7"
server_key_iv: "2d2640a873858b39dc019bd47aae5db6"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.688190 12497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:34.701442 12497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:34.702193 12497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:34.702414 12497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:34.703155 12497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:34.704141 12497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:34.704214 12497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.704291 12497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:34.704335 12497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.713505 12497 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.65:45495
I20260805 23:19:34.713529 12631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.65:45495 every 8 connection(s)
I20260805 23:19:34.714421 12497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/data/info.pb
I20260805 23:19:34.720589 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12497
I20260805 23:19:34.720732 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-0/wal/instance
I20260805 23:19:34.721076 23905 external_mini_cluster.cc:1428] Setting key bd2de47ec756c0efb199426ababb0fed
I20260805 23:19:34.723342 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.66:0
--local_ip_for_outbound_sockets=127.23.88.66
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--builtin_ntp_servers=127.23.88.84:32863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:34.725041 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.716277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.65:33515 (local address 127.23.88.126:43027)
0805 23:19:34.716417 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.716421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.717370 (+ 949us) server_negotiation.cc:407] Connection header received
0805 23:19:34.718764 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.718767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.718814 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.718886 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:34.719961 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.720704 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.722482 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.722774 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.722885 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.724425 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:34.724432 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:34.724438 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:34.724455 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:34.724482 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:34.724495 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:34.724578 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.724861 (+ 283us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.724923 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:19:34.726598 12634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.716618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43027 (local address 127.23.88.65:33515)
0805 23:19:34.717242 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.717283 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.718568 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.719014 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.719020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.719163 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:34.719756 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.719772 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.720935 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.720940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.722310 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.722324 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.722954 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.722958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.723227 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.724141 (+ 914us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:34.724142 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:34.724184 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:34.724252 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:34.724638 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:34.724655 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:34.724664 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.724866 (+ 202us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.725277 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":109,"threads_started":1}
I20260805 23:19:34.728060 12632 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43027
I20260805 23:19:34.728317 12632 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:34.728871 12632 heartbeater.cc:511] Master 127.23.88.126:43027 requested a full tablet report, sending...
I20260805 23:19:34.730414 12434 ts_manager.cc:194] Registered new tserver with Master: efc9d685aa6a44fabba9cdfc5227cb78 (127.23.88.65:45495)
I20260805 23:19:34.731709 12434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.65:33515
W20260805 23:19:34.847412 12635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:34.847714 12635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:34.847760 12635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:34.851929 12635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:34.851996 12635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:34.852105 12635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.66
I20260805 23:19:34.856972 12635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:32863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.88.66
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.66
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:34.858260 12635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:34.859826 12635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:34.869822 12640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.869880 12644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:34.869822 12642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:34.869904 12635 server_base.cc:1034] running on GCE node
I20260805 23:19:34.870508 12635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:34.871302 12635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:34.872512 12635 hybrid_clock.cc:648] HybridClock initialized: now 1785971974872488 us; error 47 us; skew 500 ppm
I20260805 23:19:34.874593 12635 webserver.cc:460] Webserver started at http://127.23.88.66:46321/ using document root <none> and password file <none>
I20260805 23:19:34.875320 12635 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:34.875423 12635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:34.875676 12635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:34.877913 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data/instance:
uuid: "2799b6956e734e85a70f616ca20b3251"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "b71004120e6f7eefb9390014df1ee5bc"
server_key_iv: "129e920e513bf61c7a49375e38134df0"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.878558 12635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance:
uuid: "2799b6956e734e85a70f616ca20b3251"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "b71004120e6f7eefb9390014df1ee5bc"
server_key_iv: "129e920e513bf61c7a49375e38134df0"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.882823 12635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260805 23:19:34.886090 12650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.887509 12635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:34.887612 12635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal
uuid: "2799b6956e734e85a70f616ca20b3251"
format_stamp: "Formatted at 2026-08-05 23:19:34 on dist-test-slave-vkjp"
server_key: "b71004120e6f7eefb9390014df1ee5bc"
server_key_iv: "129e920e513bf61c7a49375e38134df0"
server_key_version: "encryptionkey@0"
I20260805 23:19:34.887709 12635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:34.901125 12635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:34.901865 12635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:34.902030 12635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:34.902585 12635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:34.903579 12635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:34.903627 12635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.903668 12635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:34.903687 12635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:34.913722 12635 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.66:35221
I20260805 23:19:34.913743 12771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.66:35221 every 8 connection(s)
I20260805 23:19:34.914723 12635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/data/info.pb
I20260805 23:19:34.921588 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12635
I20260805 23:19:34.921782 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal/instance
I20260805 23:19:34.922099 23905 external_mini_cluster.cc:1428] Setting key 9d3a2e38244554c593132a3ef534cf96
I20260805 23:19:34.924571 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.88.67:0
--local_ip_for_outbound_sockets=127.23.88.67
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--builtin_ntp_servers=127.23.88.84:32863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260805 23:19:34.925221 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.916768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:48749 (local address 127.23.88.126:43027)
0805 23:19:34.916948 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:34.916953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:34.918033 (+ 1080us) server_negotiation.cc:407] Connection header received
0805 23:19:34.919468 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:34.919470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:34.919518 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:34.919593 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:34.920607 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.921212 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.922741 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.923055 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.923212 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.924549 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:34.924553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:34.924555 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:34.924570 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:34.924595 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:34.924606 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:34.924691 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:34.924905 (+ 214us) server_negotiation.cc:299] Negotiation successful
0805 23:19:34.924990 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260805 23:19:34.926034 12774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:34.917205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43027 (local address 127.23.88.66:48749)
0805 23:19:34.917884 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:34.917928 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:34.919251 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:34.919756 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:34.919762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:34.919858 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:34.920445 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.920458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.921463 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.921466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.922579 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:34.922590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:34.923264 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:34.923268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:34.923473 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:34.924249 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:34.924250 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:34.924294 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:34.924379 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:34.924706 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:34.924716 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:34.924725 (+ 9us) client_negotiation.cc:769] Sending connection context
0805 23:19:34.924890 (+ 165us) client_negotiation.cc:240] Negotiation successful
0805 23:19:34.925147 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":123,"threads_started":1}
I20260805 23:19:34.927419 12772 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43027
I20260805 23:19:34.927670 12772 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:34.928211 12772 heartbeater.cc:511] Master 127.23.88.126:43027 requested a full tablet report, sending...
I20260805 23:19:34.929291 12434 ts_manager.cc:194] Registered new tserver with Master: 2799b6956e734e85a70f616ca20b3251 (127.23.88.66:35221)
I20260805 23:19:34.929905 12434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.66:48749
W20260805 23:19:35.039971 12775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:35.040268 12775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:35.040338 12775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:35.044566 12775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:35.044659 12775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:35.044754 12775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.88.67
I20260805 23:19:35.049583 12775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:32863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.88.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.88.67
--webserver_port=0
--tserver_master_addrs=127.23.88.126:43027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.88.67
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:35.050822 12775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:35.052304 12775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:35.061888 12780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:35.061955 12783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:35.061892 12781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:35.062283 12775 server_base.cc:1034] running on GCE node
I20260805 23:19:35.062736 12775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:35.063432 12775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:35.064620 12775 hybrid_clock.cc:648] HybridClock initialized: now 1785971975064590 us; error 40 us; skew 500 ppm
I20260805 23:19:35.066592 12775 webserver.cc:460] Webserver started at http://127.23.88.67:34643/ using document root <none> and password file <none>
I20260805 23:19:35.067224 12775 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:35.067317 12775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:35.067548 12775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:35.069701 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data/instance:
uuid: "627c98bd72ec4e7c97119e37d85b2e96"
format_stamp: "Formatted at 2026-08-05 23:19:35 on dist-test-slave-vkjp"
server_key: "65449661884b6d63d545e3b568a31d4a"
server_key_iv: "2d13fa789b93f2a02712bf1000a4be69"
server_key_version: "encryptionkey@0"
I20260805 23:19:35.070281 12775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance:
uuid: "627c98bd72ec4e7c97119e37d85b2e96"
format_stamp: "Formatted at 2026-08-05 23:19:35 on dist-test-slave-vkjp"
server_key: "65449661884b6d63d545e3b568a31d4a"
server_key_iv: "2d13fa789b93f2a02712bf1000a4be69"
server_key_version: "encryptionkey@0"
I20260805 23:19:35.074065 12775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260805 23:19:35.076524 12789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:35.077804 12775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260805 23:19:35.077936 12775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal
uuid: "627c98bd72ec4e7c97119e37d85b2e96"
format_stamp: "Formatted at 2026-08-05 23:19:35 on dist-test-slave-vkjp"
server_key: "65449661884b6d63d545e3b568a31d4a"
server_key_iv: "2d13fa789b93f2a02712bf1000a4be69"
server_key_version: "encryptionkey@0"
I20260805 23:19:35.078050 12775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:35.104637 12775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:35.105512 12775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:35.105728 12775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:35.106362 12775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260805 23:19:35.107401 12775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260805 23:19:35.107483 12775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:35.107558 12775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260805 23:19:35.107599 12775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:35.116581 12775 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.67:40221
I20260805 23:19:35.116605 12909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.67:40221 every 8 connection(s)
I20260805 23:19:35.117518 12775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/data/info.pb
I20260805 23:19:35.122274 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12775
I20260805 23:19:35.122418 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal/instance
I20260805 23:19:35.122732 23905 external_mini_cluster.cc:1428] Setting key 4f6ebc4ba2614749ff6fc99f42893760
I20260805 23:19:35.127539 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.119311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.67:52883 (local address 127.23.88.126:43027)
0805 23:19:35.119488 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.119493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.120640 (+ 1147us) server_negotiation.cc:407] Connection header received
0805 23:19:35.122119 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.122123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.122175 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.122252 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:35.123541 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.124325 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.125653 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.125866 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.125951 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.127031 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:35.127033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:35.127036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:35.127048 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:35.127068 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:35.127121 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:35.127201 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.127345 (+ 144us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.127400 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260805 23:19:35.128404 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.119762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:43027 (local address 127.23.88.67:52883)
0805 23:19:35.120443 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.120502 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.121843 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.122484 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.122495 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.122608 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:19:35.123379 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.123399 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.124466 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.124469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.125534 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.125544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.126007 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.126009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.126152 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.126758 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0805 23:19:35.126760 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:19:35.126809 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0805 23:19:35.126878 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:19:35.127214 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:19:35.127221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:19:35.127225 (+ 4us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.127327 (+ 102us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.127553 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":116,"threads_started":1}
I20260805 23:19:35.129521 12911 heartbeater.cc:347] Connected to a master server at 127.23.88.126:43027
I20260805 23:19:35.129747 12911 heartbeater.cc:464] Registering TS with master...
I20260805 23:19:35.130275 12911 heartbeater.cc:511] Master 127.23.88.126:43027 requested a full tablet report, sending...
I20260805 23:19:35.131379 12434 ts_manager.cc:194] Registered new tserver with Master: 627c98bd72ec4e7c97119e37d85b2e96 (127.23.88.67:40221)
I20260805 23:19:35.131983 12434 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.88.67:52883
I20260805 23:19:35.136509 23905 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260805 23:19:35.142373 12475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.138478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.23.88.126:43027)
0805 23:19:35.138597 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.138602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.138773 (+ 171us) server_negotiation.cc:407] Connection header received
0805 23:19:35.138874 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.138876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.138925 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.139000 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:35.139250 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.139843 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.140847 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.141072 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.141169 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.142021 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:35.142024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:35.142026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:35.142036 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:35.142054 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:35.142061 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:35.142120 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.142214 (+ 94us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.142260 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260805 23:19:35.145051 12434 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260805 23:19:35.148240 12434 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260805 23:19:35.162988 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.157239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:40221 (local address 127.0.0.1:43464)
0805 23:19:35.157638 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.157692 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.157822 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.158911 (+ 1089us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.158917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.158940 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:35.159141 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.159148 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.160383 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.160388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.161647 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.161657 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.162428 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.162432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.162544 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.162583 (+ 39us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.162713 (+ 130us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.162831 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":27,"thread_start_us":87,"threads_started":1}
I20260805 23:19:35.163647 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.157963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.23.88.67:40221)
0805 23:19:35.158402 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.158413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.158440 (+ 27us) server_negotiation.cc:407] Connection header received
0805 23:19:35.158522 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.158527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.158750 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.158919 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:35.159255 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.160275 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.161765 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.162304 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.162427 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.162638 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.163346 (+ 708us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.163467 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":112,"threads_started":1}
I20260805 23:19:35.164065 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.157239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.66:35221 (local address 127.0.0.1:58048)
0805 23:19:35.158429 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.158448 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.158545 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.159353 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.159356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.159376 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:35.159474 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.159480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.161089 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.161093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.162494 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.162506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.163606 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.163610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.163706 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.163730 (+ 24us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.163805 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.163895 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1113,"mutex_wait_us":185,"thread_start_us":111,"threads_started":1}
I20260805 23:19:35.164116 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.158049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45495 (local address 127.0.0.1:45224)
0805 23:19:35.158746 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.158761 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.158852 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.159686 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.159689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.159707 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:35.159784 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.159789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.161341 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.161348 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.162613 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.162622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.163718 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.163724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.163815 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.163840 (+ 25us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.163896 (+ 56us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.163995 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":94,"threads_started":1}
I20260805 23:19:35.164132 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.157423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.23.88.66:35221)
0805 23:19:35.158766 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.158802 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.158830 (+ 28us) server_negotiation.cc:407] Connection header received
0805 23:19:35.158916 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.158924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.159162 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.159362 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:35.159616 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.160926 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.162664 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.163418 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.163600 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.163820 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.163873 (+ 53us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.163966 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1229,"thread_start_us":155,"threads_started":1}
I20260805 23:19:35.165437 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.158235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.23.88.65:45495)
0805 23:19:35.159159 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.159171 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.159197 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:19:35.159293 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.159302 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.159488 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.159703 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:35.159893 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.161188 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.162753 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.163569 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.164965 (+ 1396us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.165148 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.165196 (+ 48us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.165283 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":113,"threads_started":1}
I20260805 23:19:35.166724 12840 tablet_service.cc:1467] Processing CreateTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b (DEFAULT_TABLE table=test-table [id=fe9e895d7f4a4226a7369645d619837e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:35.166906 12699 tablet_service.cc:1467] Processing CreateTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b (DEFAULT_TABLE table=test-table [id=fe9e895d7f4a4226a7369645d619837e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:35.167788 12840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8059d20993bc4666bbf25ba5ae209b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:35.168054 12699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8059d20993bc4666bbf25ba5ae209b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:35.168263 12562 tablet_service.cc:1467] Processing CreateTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b (DEFAULT_TABLE table=test-table [id=fe9e895d7f4a4226a7369645d619837e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260805 23:19:35.169308 12562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8059d20993bc4666bbf25ba5ae209b9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:35.174149 12928 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Bootstrap starting.
I20260805 23:19:35.175992 12929 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Bootstrap starting.
I20260805 23:19:35.176584 12928 tablet_bootstrap.cc:654] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:35.177347 12930 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Bootstrap starting.
I20260805 23:19:35.177600 12928 log.cc:824] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:35.178206 12929 tablet_bootstrap.cc:654] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:35.179109 12929 log.cc:824] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:35.179620 12930 tablet_bootstrap.cc:654] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:35.180018 12928 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: No bootstrap required, opened a new log
I20260805 23:19:35.180233 12928 ts_tablet_manager.cc:1397] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:35.180537 12930 log.cc:824] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:35.181149 12929 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: No bootstrap required, opened a new log
I20260805 23:19:35.181353 12929 ts_tablet_manager.cc:1397] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:35.182699 12930 tablet_bootstrap.cc:492] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: No bootstrap required, opened a new log
I20260805 23:19:35.182946 12930 ts_tablet_manager.cc:1397] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260805 23:19:35.184877 12928 raft_consensus.cc:348] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.185173 12928 raft_consensus.cc:374] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:35.185237 12928 raft_consensus.cc:729] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 627c98bd72ec4e7c97119e37d85b2e96, State: Initialized, Role: FOLLOWER
I20260805 23:19:35.185070 12929 raft_consensus.cc:348] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.185320 12929 raft_consensus.cc:374] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:35.185424 12929 raft_consensus.cc:729] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2799b6956e734e85a70f616ca20b3251, State: Initialized, Role: FOLLOWER
I20260805 23:19:35.185770 12928 consensus_queue.cc:260] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.185940 12929 consensus_queue.cc:260] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.186851 12928 ts_tablet_manager.cc:1428] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260805 23:19:35.186887 12929 ts_tablet_manager.cc:1428] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260805 23:19:35.187253 12911 heartbeater.cc:503] Master 127.23.88.126:43027 was elected leader, sending a full tablet report...
I20260805 23:19:35.187252 12772 heartbeater.cc:503] Master 127.23.88.126:43027 was elected leader, sending a full tablet report...
I20260805 23:19:35.187161 12930 raft_consensus.cc:348] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.187458 12930 raft_consensus.cc:374] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:35.187528 12930 raft_consensus.cc:729] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efc9d685aa6a44fabba9cdfc5227cb78, State: Initialized, Role: FOLLOWER
I20260805 23:19:35.187943 12930 consensus_queue.cc:260] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.188606 12632 heartbeater.cc:503] Master 127.23.88.126:43027 was elected leader, sending a full tablet report...
I20260805 23:19:35.188884 12930 ts_tablet_manager.cc:1428] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260805 23:19:35.215663 12633 tablet.cc:2369] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:19:35.368546 12912 tablet.cc:2369] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260805 23:19:35.416749 12773 tablet.cc:2369] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260805 23:19:35.546445 12935 raft_consensus.cc:482] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260805 23:19:35.546718 12935 raft_consensus.cc:504] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.548019 12935 leader_election.cc:288] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 627c98bd72ec4e7c97119e37d85b2e96 (127.23.88.67:40221), efc9d685aa6a44fabba9cdfc5227cb78 (127.23.88.65:45495)
I20260805 23:19:35.553623 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.548497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:49879 (local address 127.23.88.65:45495)
0805 23:19:35.548652 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.548657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.548776 (+ 119us) server_negotiation.cc:407] Connection header received
0805 23:19:35.548958 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.548964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.549049 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.549134 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:35.549441 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.550253 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.552219 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.552872 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.553017 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.553074 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.553408 (+ 334us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.553488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260805 23:19:35.553623 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.548465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.88.66:59729 (local address 127.23.88.67:40221)
0805 23:19:35.548607 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.548613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.548979 (+ 366us) server_negotiation.cc:407] Connection header received
0805 23:19:35.549115 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.549121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.549171 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.549244 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0805 23:19:35.549511 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.550255 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.552217 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.552893 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.553017 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.553072 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.553408 (+ 336us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.553488 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:19:35.553671 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.548352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.67:40221 (local address 127.23.88.66:59729)
0805 23:19:35.548902 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.548914 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.549001 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.549274 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.549276 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.549289 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:35.549396 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.549404 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.550377 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.550381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.552042 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.552055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.553028 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.553032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.553179 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.553277 (+ 98us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.553352 (+ 75us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.553472 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":524,"thread_start_us":100,"threads_started":1}
I20260805 23:19:35.553671 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.548352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45495 (local address 127.23.88.66:49879)
0805 23:19:35.548682 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.548694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0805 23:19:35.548832 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:19:35.549154 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:19:35.549155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:19:35.549169 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0805 23:19:35.549317 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.549323 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.550377 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.550381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.552042 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:19:35.552055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.553028 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.553032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:19:35.553201 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.553269 (+ 68us) client_negotiation.cc:769] Sending connection context
0805 23:19:35.553352 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:19:35.553487 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":299,"thread_start_us":90,"threads_started":1}
I20260805 23:19:35.554399 12861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8059d20993bc4666bbf25ba5ae209b9b" candidate_uuid: "2799b6956e734e85a70f616ca20b3251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "627c98bd72ec4e7c97119e37d85b2e96" is_pre_election: true
I20260805 23:19:35.554694 12584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8059d20993bc4666bbf25ba5ae209b9b" candidate_uuid: "2799b6956e734e85a70f616ca20b3251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" is_pre_election: true
I20260805 23:19:35.554768 12861 raft_consensus.cc:2454] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2799b6956e734e85a70f616ca20b3251 in term 0.
I20260805 23:19:35.554970 12584 raft_consensus.cc:2454] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2799b6956e734e85a70f616ca20b3251 in term 0.
I20260805 23:19:35.555320 12654 leader_election.cc:302] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2799b6956e734e85a70f616ca20b3251, 627c98bd72ec4e7c97119e37d85b2e96; no voters:
I20260805 23:19:35.555670 12935 raft_consensus.cc:2781] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260805 23:19:35.555750 12935 raft_consensus.cc:482] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260805 23:19:35.555781 12935 raft_consensus.cc:3037] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:35.556877 12935 raft_consensus.cc:504] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.557286 12935 leader_election.cc:288] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [CANDIDATE]: Term 1 election: Requested vote from peers 627c98bd72ec4e7c97119e37d85b2e96 (127.23.88.67:40221), efc9d685aa6a44fabba9cdfc5227cb78 (127.23.88.65:45495)
I20260805 23:19:35.557657 12584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8059d20993bc4666bbf25ba5ae209b9b" candidate_uuid: "2799b6956e734e85a70f616ca20b3251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efc9d685aa6a44fabba9cdfc5227cb78"
I20260805 23:19:35.557663 12861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8059d20993bc4666bbf25ba5ae209b9b" candidate_uuid: "2799b6956e734e85a70f616ca20b3251" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "627c98bd72ec4e7c97119e37d85b2e96"
I20260805 23:19:35.557762 12584 raft_consensus.cc:3037] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:35.557809 12861 raft_consensus.cc:3037] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:35.558843 12584 raft_consensus.cc:2454] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2799b6956e734e85a70f616ca20b3251 in term 1.
I20260805 23:19:35.558864 12861 raft_consensus.cc:2454] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2799b6956e734e85a70f616ca20b3251 in term 1.
I20260805 23:19:35.559190 12652 leader_election.cc:302] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2799b6956e734e85a70f616ca20b3251, efc9d685aa6a44fabba9cdfc5227cb78; no voters:
I20260805 23:19:35.559374 12935 raft_consensus.cc:2781] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:35.559612 12935 raft_consensus.cc:686] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 1 LEADER]: Becoming Leader. State: Replica: 2799b6956e734e85a70f616ca20b3251, State: Running, Role: LEADER
I20260805 23:19:35.559945 12935 consensus_queue.cc:237] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } }
I20260805 23:19:35.563449 12434 catalog_manager.cc:5495] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2799b6956e734e85a70f616ca20b3251 (127.23.88.66). New cstate: current_term: 1 leader_uuid: "2799b6956e734e85a70f616ca20b3251" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2799b6956e734e85a70f616ca20b3251" member_type: VOTER last_known_addr { host: "127.23.88.66" port: 35221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 } health_report { overall_health: UNKNOWN } } }
I20260805 23:19:35.615271 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.611659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.23.88.66:35221)
0805 23:19:35.611799 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:35.611802 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:35.611815 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:19:35.611900 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:35.611903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:35.611962 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:35.612052 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:19:35.612278 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.612769 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.613619 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:35.613847 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:35.613946 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:35.614271 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:19:35.614748 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:19:35.614856 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:35.614981 (+ 125us) server_negotiation.cc:299] Negotiation successful
0805 23:19:35.615044 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:19:35.621416 12584 raft_consensus.cc:1261] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78 [term 1 FOLLOWER]: Refusing update from remote peer 2799b6956e734e85a70f616ca20b3251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:35.621416 12861 raft_consensus.cc:1261] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 1 FOLLOWER]: Refusing update from remote peer 2799b6956e734e85a70f616ca20b3251: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:19:35.622192 12935 consensus_queue.cc:1035] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efc9d685aa6a44fabba9cdfc5227cb78" member_type: VOTER last_known_addr { host: "127.23.88.65" port: 45495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:35.622437 12939 consensus_queue.cc:1035] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [LEADER]: Connected to new peer: Peer: permanent_uuid: "627c98bd72ec4e7c97119e37d85b2e96" member_type: VOTER last_known_addr { host: "127.23.88.67" port: 40221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:19:35.631328 12943 mvcc.cc:205] Tried to move back new op lower bound from 7315341212137766912 to 7315341211896418304. Current Snapshot: MvccSnapshot[applied={T|T < 7315341212137766912}]
I20260805 23:19:35.641032 12433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:35.641208 12433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260805 23:19:35.644030 12433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55f5ba355c234b5e8251f2d3abaf9c4e: Sending DeleteTablet for 3 replicas of tablet 8059d20993bc4666bbf25ba5ae209b9b
I20260805 23:19:35.645020 12699 tablet_service.cc:1514] Processing DeleteTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:35 UTC) from {username='slave'} at 127.0.0.1:58048
I20260805 23:19:35.645145 12562 tablet_service.cc:1514] Processing DeleteTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:35 UTC) from {username='slave'} at 127.0.0.1:45224
I20260805 23:19:35.645144 12840 tablet_service.cc:1514] Processing DeleteTablet for tablet 8059d20993bc4666bbf25ba5ae209b9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-05 23:19:35 UTC) from {username='slave'} at 127.0.0.1:43464
I20260805 23:19:35.645618 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12497
I20260805 23:19:35.645686 12953 tablet_replica.cc:330] T 8059d20993bc4666bbf25ba5ae209b9b P efc9d685aa6a44fabba9cdfc5227cb78: stopping tablet replica
I20260805 23:19:35.646625 12954 tablet_replica.cc:330] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: stopping tablet replica
I20260805 23:19:35.647274 12954 raft_consensus.cc:2229] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 1 LEADER]: Raft consensus shutting down.
I20260805 23:19:35.647862 12954 raft_consensus.cc:2258] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:35.650002 12954 ts_tablet_manager.cc:1905] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260805 23:19:35.653532 12955 tablet_replica.cc:330] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: stopping tablet replica
I20260805 23:19:35.653699 12954 ts_tablet_manager.cc:1918] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:35.653815 12954 log.cc:1196] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-1/wal/wals/8059d20993bc4666bbf25ba5ae209b9b
I20260805 23:19:35.654109 12955 raft_consensus.cc:2229] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260805 23:19:35.654155 12954 ts_tablet_manager.cc:1939] T 8059d20993bc4666bbf25ba5ae209b9b P 2799b6956e734e85a70f616ca20b3251: Deleting consensus metadata
I20260805 23:19:35.654544 12955 raft_consensus.cc:2258] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260805 23:19:35.655586 12417 catalog_manager.cc:4841] TS 2799b6956e734e85a70f616ca20b3251 (127.23.88.66:35221): tablet 8059d20993bc4666bbf25ba5ae209b9b (table test-table [id=fe9e895d7f4a4226a7369645d619837e]) successfully deleted
I20260805 23:19:35.656288 12955 ts_tablet_manager.cc:1905] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260805 23:19:35.658522 12417 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.88.65:45495 (error 108)
I20260805 23:19:35.658988 12955 ts_tablet_manager.cc:1918] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260805 23:19:35.659121 12955 log.cc:1196] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Deleting WAL directory at /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785971715702001-23905-0/minicluster-data/ts-2/wal/wals/8059d20993bc4666bbf25ba5ae209b9b
I20260805 23:19:35.659472 12955 ts_tablet_manager.cc:1939] T 8059d20993bc4666bbf25ba5ae209b9b P 627c98bd72ec4e7c97119e37d85b2e96: Deleting consensus metadata
I20260805 23:19:35.659780 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:35.659460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.65:45495 (local address 127.0.0.1:45226)
0805 23:19:35.659599 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0805 23:19:35.659682 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260805 23:19:35.659780 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12635
I20260805 23:19:35.661417 12419 catalog_manager.cc:4841] TS 627c98bd72ec4e7c97119e37d85b2e96 (127.23.88.67:40221): tablet 8059d20993bc4666bbf25ba5ae209b9b (table test-table [id=fe9e895d7f4a4226a7369645d619837e]) successfully deleted
W20260805 23:19:35.661713 12417 catalog_manager.cc:4584] TS efc9d685aa6a44fabba9cdfc5227cb78 (127.23.88.65:45495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8059d20993bc4666bbf25ba5ae209b9b: Network error: Client connection negotiation failed: client connection to 127.23.88.65:45495: connect: Connection refused (error 111)
I20260805 23:19:35.666345 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12775
I20260805 23:19:35.672533 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12400
2026-08-05T23:19:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1418 ms)
[----------] 6 tests from EncryptionPolicyTest (11135 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:35 dist-test-slave-vkjp krb5kdc[12959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:35 dist-test-slave-vkjp krb5kdc[12959](info): set up 2 sockets
Aug 05 23:19:35 dist-test-slave-vkjp krb5kdc[12959](info): commencing operation
krb5kdc: starting...
W20260805 23:19:36.745213 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:36 dist-test-slave-vkjp krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:36.875531 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37913
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:36.993135 12973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:36.993578 12973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:36.993669 12973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:37.001333 12973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:37.001412 12973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:37.001433 12973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:37.001451 12973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:37.001494 12973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:37.007297 12973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:37.008460 12973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:37.009536 12973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:37.018369 12978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:37.018360 12981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:37.018338 12979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:37.018791 12973 server_base.cc:1034] running on GCE node
I20260805 23:19:37.019534 12973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:37.020699 12973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:37.021843 12973 hybrid_clock.cc:648] HybridClock initialized: now 1785971977021817 us; error 42 us; skew 500 ppm
Aug 05 23:19:37 dist-test-slave-vkjp krb5kdc[12959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971977, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:37.029487 12973 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:37.030527 12973 webserver.cc:460] Webserver started at http://127.0.0.1:35219/ using document root <none> and password file <none>
I20260805 23:19:37.031203 12973 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:37.031275 12973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:37.031522 12973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:37.033690 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "ae7297f377a5402abf4920c8c3c20403"
format_stamp: "Formatted at 2026-08-05 23:19:37 on dist-test-slave-vkjp"
server_key: "ac3eb5eb08e8c4e14dac74fc847c4488"
server_key_iv: "d410968196195c38bcca7ab77887f987"
server_key_version: "encryptionkey@0"
I20260805 23:19:37.034279 12973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "ae7297f377a5402abf4920c8c3c20403"
format_stamp: "Formatted at 2026-08-05 23:19:37 on dist-test-slave-vkjp"
server_key: "ac3eb5eb08e8c4e14dac74fc847c4488"
server_key_iv: "d410968196195c38bcca7ab77887f987"
server_key_version: "encryptionkey@0"
I20260805 23:19:37.038112 12973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:19:37.040669 12989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:37.042012 12973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:19:37.042174 12973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "ae7297f377a5402abf4920c8c3c20403"
format_stamp: "Formatted at 2026-08-05 23:19:37 on dist-test-slave-vkjp"
server_key: "ac3eb5eb08e8c4e14dac74fc847c4488"
server_key_iv: "d410968196195c38bcca7ab77887f987"
server_key_version: "encryptionkey@0"
I20260805 23:19:37.042292 12973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:37.070669 12973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:37.073907 12973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:37.074146 12973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:37.081849 12973 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37913
I20260805 23:19:37.081867 13043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37913 every 8 connection(s)
I20260805 23:19:37.082947 12973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:37.085829 13044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:37.091478 13044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403: Bootstrap starting.
I20260805 23:19:37.093034 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 12973
I20260805 23:19:37.093191 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:37.093474 23905 external_mini_cluster.cc:1428] Setting key 86149fc122c2eecb67865ed6ae566ea2
I20260805 23:19:37.094173 13044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:37.095021 13044 log.cc:824] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:37.097213 13044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403: No bootstrap required, opened a new log
I20260805 23:19:37.101591 13044 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260805 23:19:37.101918 13044 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:37.101969 13044 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae7297f377a5402abf4920c8c3c20403, State: Initialized, Role: FOLLOWER
I20260805 23:19:37.102437 13044 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260805 23:19:37.102571 13044 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:37.102658 13044 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 05 23:19:37 dist-test-slave-vkjp krb5kdc[12959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971976, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:19:37.102777 13044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:37.104038 13044 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260805 23:19:37.104440 13044 leader_election.cc:302] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae7297f377a5402abf4920c8c3c20403; no voters:
I20260805 23:19:37.104871 13044 leader_election.cc:288] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:37.105003 13049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:37.105252 13049 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [term 1 LEADER]: Becoming Leader. State: Replica: ae7297f377a5402abf4920c8c3c20403, State: Running, Role: LEADER
I20260805 23:19:37.105559 13049 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } }
I20260805 23:19:37.106227 13044 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:37.107386 13051 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae7297f377a5402abf4920c8c3c20403. Latest consensus state: current_term: 1 leader_uuid: "ae7297f377a5402abf4920c8c3c20403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } } }
I20260805 23:19:37.107515 13051 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:37.107795 13052 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae7297f377a5402abf4920c8c3c20403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae7297f377a5402abf4920c8c3c20403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37913 } } }
I20260805 23:19:37.107897 13052 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:37.108000 13059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:37.109529 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:37.094732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43353 (local address 127.0.0.1:37913)
0805 23:19:37.095227 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:37.095244 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:37.095280 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:19:37.096131 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:37.096151 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:37.096470 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:37.096782 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:37.097666 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:37.098611 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:37.099852 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:37.100319 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:37.100435 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:37.103483 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:37.103489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:37.103505 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:37.103536 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:37.105533 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:37.106157 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:37.106163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:37.106168 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:37.106304 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:37.106675 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:37.106679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:37.106681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:37.107312 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:37.107548 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:37.108349 (+ 801us) server_negotiation.cc:299] Negotiation successful
0805 23:19:37.108576 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":149,"threads_started":1}
I20260805 23:19:37.111485 13059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:37.112874 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:37.120378 13059 catalog_manager.cc:1347] Generated new cluster ID: ba77c1ce0dbe4176b38cc3ed64d75c66
I20260805 23:19:37.120452 13059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:37.124894 13047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:37.115218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43791 (local address 127.0.0.1:37913)
0805 23:19:37.115361 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:37.115366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:37.116355 (+ 989us) server_negotiation.cc:407] Connection header received
0805 23:19:37.116500 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:37.116503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:37.116554 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:37.116634 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:37.117510 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:37.118143 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:37.119110 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:37.119356 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:37.119484 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:37.121205 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:37.121209 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:37.121212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:37.121239 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:37.122971 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:37.123535 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:37.123539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:37.123541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:37.123632 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:37.123974 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:37.123978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:37.123980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:37.124204 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:37.124357 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:37.124627 (+ 270us) server_negotiation.cc:299] Negotiation successful
0805 23:19:37.124732 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:19:37.152981 13059 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:37.153996 13059 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:37.159812 13059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae7297f377a5402abf4920c8c3c20403: Generated new TSK 0
I20260805 23:19:37.160633 13059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:37.174232 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 12973
W20260805 23:19:37.181084 12957 connection.cc:535] client connection to 127.0.0.1:37913 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37913): Connection reset by peer (error 104)
2026-08-05T23:19:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:37 dist-test-slave-vkjp krb5kdc[13077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:37 dist-test-slave-vkjp krb5kdc[13077](info): set up 2 sockets
Aug 05 23:19:37 dist-test-slave-vkjp krb5kdc[13077](info): commencing operation
krb5kdc: starting...
W20260805 23:19:38.249462 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:38 dist-test-slave-vkjp krb5kdc[13077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:38.387487 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46333
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:38.505586 13091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:38.505887 13091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:38.505934 13091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:38.510097 13091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:38.510180 13091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:38.510200 13091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:38.510218 13091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:38.510233 13091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:38.515985 13091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46333
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:38.517143 13091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:38.518181 13091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:38.527679 13096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:38.527683 13097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:38.527679 13099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:38.527992 13091 server_base.cc:1034] running on GCE node
I20260805 23:19:38.528934 13091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:38.530169 13091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:38.531386 13091 hybrid_clock.cc:648] HybridClock initialized: now 1785971978531356 us; error 43 us; skew 500 ppm
Aug 05 23:19:38 dist-test-slave-vkjp krb5kdc[13077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971978, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:38.534435 13091 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:38.535466 13091 webserver.cc:460] Webserver started at http://127.0.0.1:32911/ using document root <none> and password file <none>
I20260805 23:19:38.536022 13091 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:38.536087 13091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:38.536263 13091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:38.538452 13091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "2d9028e63c0043568a2230f2013adcff"
format_stamp: "Formatted at 2026-08-05 23:19:38 on dist-test-slave-vkjp"
server_key: "1c5aea8578dd5c53672363c0a53f8657"
server_key_iv: "d8f2f7da3ff900ecc41fc98aef21e761"
server_key_version: "encryptionkey@0"
I20260805 23:19:38.539014 13091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "2d9028e63c0043568a2230f2013adcff"
format_stamp: "Formatted at 2026-08-05 23:19:38 on dist-test-slave-vkjp"
server_key: "1c5aea8578dd5c53672363c0a53f8657"
server_key_iv: "d8f2f7da3ff900ecc41fc98aef21e761"
server_key_version: "encryptionkey@0"
I20260805 23:19:38.543547 13091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:19:38.546283 13106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:38.547760 13091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:38.547915 13091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "2d9028e63c0043568a2230f2013adcff"
format_stamp: "Formatted at 2026-08-05 23:19:38 on dist-test-slave-vkjp"
server_key: "1c5aea8578dd5c53672363c0a53f8657"
server_key_iv: "d8f2f7da3ff900ecc41fc98aef21e761"
server_key_version: "encryptionkey@0"
I20260805 23:19:38.548050 13091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:38.570313 13091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:38.573832 13091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:38.574080 13091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:38.581851 13091 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46161
I20260805 23:19:38.581871 13161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46161 every 8 connection(s)
I20260805 23:19:38.582861 13091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:38.584862 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13091
I20260805 23:19:38.585044 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:38.585366 23905 external_mini_cluster.cc:1428] Setting key 3670c0af52f776794d0949ea8f15ac7d
I20260805 23:19:38.585976 13162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:38.592906 13162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff: Bootstrap starting.
I20260805 23:19:38.594786 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:38.586819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58423 (local address 127.0.0.1:46161)
0805 23:19:38.587303 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:38.587318 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:38.587354 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:19:38.588027 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:38.588043 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:38.588365 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:38.588651 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:38.589068 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:38.590096 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:38.591345 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:38.591837 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:38.591961 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:38.592958 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:38.592963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:38.592975 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:38.593432 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:38.593532 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:38.593558 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:38.593666 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:38.593874 (+ 208us) server_negotiation.cc:299] Negotiation successful
0805 23:19:38.594032 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":176,"threads_started":1}
I20260805 23:19:38.595439 13162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:38.596204 13162 log.cc:824] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:38.597865 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:38.598469 13162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff: No bootstrap required, opened a new log
I20260805 23:19:38.602824 13162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260805 23:19:38.603175 13162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:38.603243 13162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d9028e63c0043568a2230f2013adcff, State: Initialized, Role: FOLLOWER
I20260805 23:19:38.603722 13162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260805 23:19:38.603911 13162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:38.603986 13162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:38.604089 13162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:38.604377 13165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:38.600055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49449 (local address 127.0.0.1:46161)
0805 23:19:38.600162 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:38.600166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:38.600350 (+ 184us) server_negotiation.cc:407] Connection header received
0805 23:19:38.600436 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:38.600440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:38.600492 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:38.600594 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:38.600839 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:38.601634 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:38.602492 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:38.602812 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:38.602924 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:38.603873 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:38.603876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:38.603878 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:38.603890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:38.603917 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:38.603927 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:38.604037 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:38.604169 (+ 132us) server_negotiation.cc:299] Negotiation successful
0805 23:19:38.604209 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260805 23:19:38.605335 13162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260805 23:19:38.605734 13162 leader_election.cc:302] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d9028e63c0043568a2230f2013adcff; no voters:
I20260805 23:19:38.606009 13162 leader_election.cc:288] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:38.606189 13172 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:38.606503 13172 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [term 1 LEADER]: Becoming Leader. State: Replica: 2d9028e63c0043568a2230f2013adcff, State: Running, Role: LEADER
I20260805 23:19:38.606840 13172 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } }
I20260805 23:19:38.607379 13162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:38.608623 13174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d9028e63c0043568a2230f2013adcff. Latest consensus state: current_term: 1 leader_uuid: "2d9028e63c0043568a2230f2013adcff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260805 23:19:38.608745 13174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:38.609160 13181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:38.609184 13173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d9028e63c0043568a2230f2013adcff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d9028e63c0043568a2230f2013adcff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46161 } } }
I20260805 23:19:38.609341 13173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:38.612571 13181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:38.618260 13181 catalog_manager.cc:1347] Generated new cluster ID: 833c49c66bbd433d87b05fa360726541
I20260805 23:19:38.618340 13181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:38.636128 13181 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:38.637717 13181 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:38.649709 13181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d9028e63c0043568a2230f2013adcff: Generated new TSK 0
I20260805 23:19:38.650477 13181 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:38.660452 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13091
W20260805 23:19:38.667225 13075 connection.cc:535] client connection to 127.0.0.1:46161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46161): Connection reset by peer (error 104)
2026-08-05T23:19:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1486 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:38 dist-test-slave-vkjp krb5kdc[13193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:38 dist-test-slave-vkjp krb5kdc[13193](info): set up 2 sockets
Aug 05 23:19:38 dist-test-slave-vkjp krb5kdc[13193](info): commencing operation
krb5kdc: starting...
W20260805 23:19:39.744724 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:39 dist-test-slave-vkjp krb5kdc[13193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971979, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:39.875680 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:39.987968 13207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:39.988237 13207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:39.988281 13207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:39.992298 13207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:39.992363 13207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:39.992383 13207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:39.992399 13207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:39.992414 13207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:39.997785 13207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:39.998817 13207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:39.999861 13207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:40.008983 13212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:40.009068 13215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:40.009202 13213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:40.009536 13207 server_base.cc:1034] running on GCE node
I20260805 23:19:40.010145 13207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:40.011487 13207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:40.012704 13207 hybrid_clock.cc:648] HybridClock initialized: now 1785971980012688 us; error 38 us; skew 500 ppm
Aug 05 23:19:40 dist-test-slave-vkjp krb5kdc[13193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971980, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:40.015631 13207 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:40.016590 13207 webserver.cc:460] Webserver started at http://127.0.0.1:34643/ using document root <none> and password file <none>
I20260805 23:19:40.017134 13207 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:40.017198 13207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:40.017417 13207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:40.019570 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "faf2b240c5fe4f0892bcd28da482ba30"
format_stamp: "Formatted at 2026-08-05 23:19:40 on dist-test-slave-vkjp"
server_key: "f7f7954eae3efbf777cd6c3ab2adf5ad"
server_key_iv: "67b869f548585eb278011432e40f963b"
server_key_version: "encryptionkey@0"
I20260805 23:19:40.020110 13207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "faf2b240c5fe4f0892bcd28da482ba30"
format_stamp: "Formatted at 2026-08-05 23:19:40 on dist-test-slave-vkjp"
server_key: "f7f7954eae3efbf777cd6c3ab2adf5ad"
server_key_iv: "67b869f548585eb278011432e40f963b"
server_key_version: "encryptionkey@0"
I20260805 23:19:40.024001 13207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:40.026448 13222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:40.027770 13207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:40.027873 13207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "faf2b240c5fe4f0892bcd28da482ba30"
format_stamp: "Formatted at 2026-08-05 23:19:40 on dist-test-slave-vkjp"
server_key: "f7f7954eae3efbf777cd6c3ab2adf5ad"
server_key_iv: "67b869f548585eb278011432e40f963b"
server_key_version: "encryptionkey@0"
I20260805 23:19:40.027977 13207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:40.032716 13207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:40.035601 13207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:40.035751 13207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:40.042987 13207 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35037
I20260805 23:19:40.043008 13277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35037 every 8 connection(s)
I20260805 23:19:40.043967 13207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:40.046705 13278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:40.052182 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13207
I20260805 23:19:40.052320 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:40.052558 13278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30: Bootstrap starting.
I20260805 23:19:40.052656 23905 external_mini_cluster.cc:1428] Setting key ddddbf648414d1dd5de746109887df87
I20260805 23:19:40.055240 13278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:40.055990 13278 log.cc:824] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:40.057955 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:40.054056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45909 (local address 127.0.0.1:35037)
0805 23:19:40.054511 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:40.054533 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:40.054569 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:19:40.055282 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:40.055324 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:40.055621 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:40.055945 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:40.056740 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:40.056748 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:40.056765 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:40.056782 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:40.056856 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:40.056893 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:40.057026 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:40.057127 (+ 101us) server_negotiation.cc:299] Negotiation successful
0805 23:19:40.057285 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":137,"threads_started":1}
I20260805 23:19:40.058207 13278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30: No bootstrap required, opened a new log
I20260805 23:19:40.060698 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:40.062083 13278 raft_consensus.cc:348] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } }
I20260805 23:19:40.062320 13278 raft_consensus.cc:374] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:40.062397 13278 raft_consensus.cc:729] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faf2b240c5fe4f0892bcd28da482ba30, State: Initialized, Role: FOLLOWER
I20260805 23:19:40.062884 13278 consensus_queue.cc:260] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } }
I20260805 23:19:40.063073 13278 raft_consensus.cc:388] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:40.063176 13278 raft_consensus.cc:482] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:40.063279 13278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:40.064378 13278 raft_consensus.cc:504] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } }
I20260805 23:19:40.064579 13282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:40.062778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56207 (local address 127.0.0.1:35037)
0805 23:19:40.062902 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:40.062908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:40.063130 (+ 222us) server_negotiation.cc:407] Connection header received
0805 23:19:40.063334 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:40.063338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:40.063398 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:40.063474 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:40.064163 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:40.064166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:40.064168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:40.064179 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:40.064213 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:40.064223 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:40.064313 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:40.064386 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 23:19:40.064426 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260805 23:19:40.064803 13278 leader_election.cc:302] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faf2b240c5fe4f0892bcd28da482ba30; no voters:
I20260805 23:19:40.065146 13278 leader_election.cc:288] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:40.065316 13289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:40.065598 13289 raft_consensus.cc:686] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [term 1 LEADER]: Becoming Leader. State: Replica: faf2b240c5fe4f0892bcd28da482ba30, State: Running, Role: LEADER
I20260805 23:19:40.065943 13289 consensus_queue.cc:237] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } }
I20260805 23:19:40.066448 13278 sys_catalog.cc:564] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:40.067090 13289 sys_catalog.cc:455] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faf2b240c5fe4f0892bcd28da482ba30. Latest consensus state: current_term: 1 leader_uuid: "faf2b240c5fe4f0892bcd28da482ba30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } } }
I20260805 23:19:40.067312 13289 sys_catalog.cc:458] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:40.067569 13291 sys_catalog.cc:455] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faf2b240c5fe4f0892bcd28da482ba30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faf2b240c5fe4f0892bcd28da482ba30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35037 } } }
I20260805 23:19:40.067694 13291 sys_catalog.cc:458] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:40.067850 13298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:40.071122 13298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:40.077646 13298 catalog_manager.cc:1347] Generated new cluster ID: 9cc8f299650642d6af069574578bc2f9
I20260805 23:19:40.077754 13298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:40.090016 13298 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:40.090371 13298 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:40.101836 13298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faf2b240c5fe4f0892bcd28da482ba30: Generated new TSK 0
I20260805 23:19:40.102726 13298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:40.109946 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13207
2026-08-05T23:19:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:40 dist-test-slave-vkjp krb5kdc[13310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:40 dist-test-slave-vkjp krb5kdc[13310](info): set up 2 sockets
Aug 05 23:19:40 dist-test-slave-vkjp krb5kdc[13310](info): commencing operation
krb5kdc: starting...
W20260805 23:19:41.192669 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:41.326112 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:41.440383 13325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:41.440716 13325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:41.440793 13325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:41.444921 13325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:41.445043 13325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:41.445084 13325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:41.445122 13325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:41.445155 13325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:41.450819 13325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37329
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:41.452093 13325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:41.453173 13325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:41.462118 13331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:41.462245 13333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:41.462118 13330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:41.462482 13325 server_base.cc:1034] running on GCE node
I20260805 23:19:41.463176 13325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:41.464334 13325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:41.465507 13325 hybrid_clock.cc:648] HybridClock initialized: now 1785971981465488 us; error 36 us; skew 500 ppm
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971981, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:41.474251 13325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:41.475420 13325 webserver.cc:460] Webserver started at http://127.0.0.1:42779/ using document root <none> and password file <none>
I20260805 23:19:41.476049 13325 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:41.476121 13325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:41.476398 13325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:41.478634 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "686023220f3d44f6b06469e0bbdd637d"
format_stamp: "Formatted at 2026-08-05 23:19:41 on dist-test-slave-vkjp"
server_key: "9a2c777cf94bb33187df42146ef2c928"
server_key_iv: "ae622eea7d7edd152fbe64c2871055fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:41.479310 13325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "686023220f3d44f6b06469e0bbdd637d"
format_stamp: "Formatted at 2026-08-05 23:19:41 on dist-test-slave-vkjp"
server_key: "9a2c777cf94bb33187df42146ef2c928"
server_key_iv: "ae622eea7d7edd152fbe64c2871055fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:41.483491 13325 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:41.486042 13341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:41.487411 13325 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:41.487555 13325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "686023220f3d44f6b06469e0bbdd637d"
format_stamp: "Formatted at 2026-08-05 23:19:41 on dist-test-slave-vkjp"
server_key: "9a2c777cf94bb33187df42146ef2c928"
server_key_iv: "ae622eea7d7edd152fbe64c2871055fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:41.487677 13325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:41.532701 13325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:41.536528 13325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:41.536753 13325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:41.544119 13325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37329
I20260805 23:19:41.544149 13394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37329 every 8 connection(s)
I20260805 23:19:41.545118 13325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:41.547953 13395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:41.553589 13395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d: Bootstrap starting.
I20260805 23:19:41.553798 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13325
I20260805 23:19:41.553952 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:41.554255 23905 external_mini_cluster.cc:1428] Setting key b0065d56d361991badf5683e44d8e302
I20260805 23:19:41.556346 13395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:41.557124 13395 log.cc:824] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:41.559612 13395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d: No bootstrap required, opened a new log
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:19:41.564534 13395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260805 23:19:41.564826 13395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:41.564894 13395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 686023220f3d44f6b06469e0bbdd637d, State: Initialized, Role: FOLLOWER
I20260805 23:19:41.565351 13395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260805 23:19:41.565513 13395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:41.565604 13395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:41.565728 13395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:41.566715 13395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260805 23:19:41.567126 13395 leader_election.cc:302] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 686023220f3d44f6b06469e0bbdd637d; no voters:
I20260805 23:19:41.567531 13395 leader_election.cc:288] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:41.567610 13400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:41.567831 13400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [term 1 LEADER]: Becoming Leader. State: Replica: 686023220f3d44f6b06469e0bbdd637d, State: Running, Role: LEADER
I20260805 23:19:41.568207 13400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } }
I20260805 23:19:41.568935 13395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:41.570006 13401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "686023220f3d44f6b06469e0bbdd637d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } } }
I20260805 23:19:41.570154 13401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:41.570469 13402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 686023220f3d44f6b06469e0bbdd637d. Latest consensus state: current_term: 1 leader_uuid: "686023220f3d44f6b06469e0bbdd637d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "686023220f3d44f6b06469e0bbdd637d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37329 } } }
I20260805 23:19:41.570566 13402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:41.570645 13410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:41.571118 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:41.555658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57395 (local address 127.0.0.1:37329)
0805 23:19:41.556355 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:41.556371 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:41.556409 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:19:41.557135 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:41.557152 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:41.557643 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:41.558051 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:41.558937 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:41.560094 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:41.561288 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:41.561791 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:41.561918 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:41.564984 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:41.564992 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:41.565012 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:41.565055 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:41.567182 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:41.567746 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:41.567760 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:41.567764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:41.567942 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:41.568276 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:41.568279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:41.568281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:41.568720 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:41.568947 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:41.569355 (+ 408us) server_negotiation.cc:299] Negotiation successful
0805 23:19:41.569855 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":144,"threads_started":1}
I20260805 23:19:41.574020 13410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:41.574841 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:41.580879 13410 catalog_manager.cc:1347] Generated new cluster ID: e7cc5d200c0a4fa5a81feeef38cf99fb
I20260805 23:19:41.580964 13410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:41.589324 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:41.577098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38115 (local address 127.0.0.1:37329)
0805 23:19:41.577201 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:41.577206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:41.579525 (+ 2319us) server_negotiation.cc:407] Connection header received
0805 23:19:41.579746 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:41.579750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:41.579811 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:41.579911 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:41.580853 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:41.581729 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:41.583036 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:41.583410 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:41.583545 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:41.585119 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:41.585123 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:41.585126 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:41.585165 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:41.586882 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:41.587530 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:41.587536 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:41.587539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:41.587640 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:41.588019 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:41.588026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:41.588029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:41.588302 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:41.588426 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:41.588900 (+ 474us) server_negotiation.cc:299] Negotiation successful
0805 23:19:41.589070 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":28}
I20260805 23:19:41.597743 13410 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:41.598884 13410 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:41.607594 13410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 686023220f3d44f6b06469e0bbdd637d: Generated new TSK 0
I20260805 23:19:41.608419 13410 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:41.613704 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13325
2026-08-05T23:19:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13429](info): set up 2 sockets
Aug 05 23:19:41 dist-test-slave-vkjp krb5kdc[13429](info): commencing operation
krb5kdc: starting...
W20260805 23:19:42.694226 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:42 dist-test-slave-vkjp krb5kdc[13429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971982, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:42.834545 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:42.958225 13442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:42.958573 13442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:42.958666 13442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:42.963007 13442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:42.963155 13442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:42.963204 13442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:42.963275 13442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:42.963316 13442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:42.968887 13442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:42.970103 13442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:42.971252 13442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:42.980280 13448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:42.980312 13451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:42.980312 13447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:42.980370 13442 server_base.cc:1034] running on GCE node
I20260805 23:19:42.981164 13442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:42.982348 13442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:42.983520 13442 hybrid_clock.cc:648] HybridClock initialized: now 1785971982983500 us; error 32 us; skew 500 ppm
Aug 05 23:19:42 dist-test-slave-vkjp krb5kdc[13429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971982, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:42.991940 13442 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:42.993093 13442 webserver.cc:460] Webserver started at http://127.0.0.1:40813/ using document root <none> and password file <none>
I20260805 23:19:42.993803 13442 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:42.993880 13442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:42.994115 13442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:42.996548 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "dcac9e6294214fd291f1379371b58f3d"
format_stamp: "Formatted at 2026-08-05 23:19:42 on dist-test-slave-vkjp"
server_key: "218ee1a7845cb74a21d3f8a2dcd68574"
server_key_iv: "1b6171de5347ecbb8e984a276e897015"
server_key_version: "encryptionkey@0"
I20260805 23:19:42.997182 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "dcac9e6294214fd291f1379371b58f3d"
format_stamp: "Formatted at 2026-08-05 23:19:42 on dist-test-slave-vkjp"
server_key: "218ee1a7845cb74a21d3f8a2dcd68574"
server_key_iv: "1b6171de5347ecbb8e984a276e897015"
server_key_version: "encryptionkey@0"
I20260805 23:19:43.001338 13442 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:43.003916 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:43.005194 13442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:43.005317 13442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "dcac9e6294214fd291f1379371b58f3d"
format_stamp: "Formatted at 2026-08-05 23:19:42 on dist-test-slave-vkjp"
server_key: "218ee1a7845cb74a21d3f8a2dcd68574"
server_key_iv: "1b6171de5347ecbb8e984a276e897015"
server_key_version: "encryptionkey@0"
I20260805 23:19:43.005422 13442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:43.021234 13442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:43.024645 13442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:43.024868 13442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:43.032114 13442 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39931
I20260805 23:19:43.032173 13512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39931 every 8 connection(s)
I20260805 23:19:43.033073 13442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:43.035892 13513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:43.041625 13513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: Bootstrap starting.
I20260805 23:19:43.042699 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13442
I20260805 23:19:43.042865 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:43.043222 23905 external_mini_cluster.cc:1428] Setting key 0ba4cb8dae769d600bf9d288f6fcaf5e
I20260805 23:19:43.044615 13513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:43.045358 13513 log.cc:824] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:43.047642 13513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: No bootstrap required, opened a new log
I20260805 23:19:43.051626 13513 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260805 23:19:43.051944 13513 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:43.052008 13513 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcac9e6294214fd291f1379371b58f3d, State: Initialized, Role: FOLLOWER
I20260805 23:19:43.052585 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:43.044376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34147 (local address 127.0.0.1:39931)
0805 23:19:43.044876 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:43.044892 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:43.044926 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:19:43.045645 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:43.045664 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:43.046120 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:43.046493 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:43.046908 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:43.047995 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:43.048914 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:43.049387 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:43.049572 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:43.050442 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:43.050453 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:43.050467 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:43.050855 (+ 388us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:43.050915 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:43.050937 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:43.051051 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:43.051238 (+ 187us) server_negotiation.cc:299] Negotiation successful
0805 23:19:43.051477 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":148,"threads_started":1}
I20260805 23:19:43.052527 13513 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260805 23:19:43.052697 13513 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:43.052763 13513 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:43.052882 13513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:43.054165 13513 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260805 23:19:43.054581 13513 leader_election.cc:302] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcac9e6294214fd291f1379371b58f3d; no voters:
I20260805 23:19:43.055044 13513 leader_election.cc:288] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:43.055151 13520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:43.055472 13520 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [term 1 LEADER]: Becoming Leader. State: Replica: dcac9e6294214fd291f1379371b58f3d, State: Running, Role: LEADER
I20260805 23:19:43.055749 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:43.055835 13520 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260805 23:19:43.056447 13513 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:43.057560 13521 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcac9e6294214fd291f1379371b58f3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } } }
I20260805 23:19:43.057700 13521 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:43.058090 13523 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcac9e6294214fd291f1379371b58f3d. Latest consensus state: current_term: 1 leader_uuid: "dcac9e6294214fd291f1379371b58f3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcac9e6294214fd291f1379371b58f3d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } } }
I20260805 23:19:43.058244 13523 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d [sys.catalog]: This master's current role is: LEADER
W20260805 23:19:43.061668 13541 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:19:43.061777 13541 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:19:43.061904 13534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:43.062677 13534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:43.063454 13517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:43.058159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59987 (local address 127.0.0.1:39931)
0805 23:19:43.058271 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:43.058276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:43.059177 (+ 901us) server_negotiation.cc:407] Connection header received
0805 23:19:43.059371 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:43.059374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:43.059420 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:43.059504 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:43.059774 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:43.060347 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:43.061489 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:43.061832 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:43.061907 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:43.062935 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:43.062939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:43.062942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:43.062956 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:43.062985 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:43.063001 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:43.063144 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:43.063258 (+ 114us) server_negotiation.cc:299] Negotiation successful
0805 23:19:43.063321 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:19:43.068488 13534 catalog_manager.cc:1347] Generated new cluster ID: a1c7f214a9fe4e13bb37ff0f28e2e8b5
I20260805 23:19:43.068559 13534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:43.087747 13534 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:43.088689 13534 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:43.094010 13534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcac9e6294214fd291f1379371b58f3d: Generated new TSK 0
I20260805 23:19:43.094781 13534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:43.103525 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13442
2026-08-05T23:19:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:43 dist-test-slave-vkjp krb5kdc[13547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:43 dist-test-slave-vkjp krb5kdc[13547](info): set up 2 sockets
Aug 05 23:19:43 dist-test-slave-vkjp krb5kdc[13547](info): commencing operation
krb5kdc: starting...
W20260805 23:19:44.180835 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:44 dist-test-slave-vkjp krb5kdc[13547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:44.310467 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:44.424831 13561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:44.425149 13561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:44.425231 13561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:44.429328 13561 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:44.429423 13561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:44.429461 13561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:44.429520 13561 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:44.429558 13561 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:44.434835 13561 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:44.435949 13561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:44.436969 13561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260805 23:19:44.445875 13561 server_base.cc:1034] running on GCE node
W20260805 23:19:44.445813 13566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:44.445886 13567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:44.445787 13569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:44.446650 13561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:44.447825 13561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:44.449015 13561 hybrid_clock.cc:648] HybridClock initialized: now 1785971984449003 us; error 24 us; skew 500 ppm
Aug 05 23:19:44 dist-test-slave-vkjp krb5kdc[13547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971984, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:44.451902 13561 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:44.452869 13561 webserver.cc:460] Webserver started at http://127.0.0.1:35133/ using document root <none> and password file <none>
I20260805 23:19:44.453527 13561 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:44.453626 13561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:44.453857 13561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:44.456055 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "dc76795b414c45f28df99d95d5049a70"
format_stamp: "Formatted at 2026-08-05 23:19:44 on dist-test-slave-vkjp"
server_key: "13b5a73fd04fbffc10a7c3bfb573994a"
server_key_iv: "763c52605b2807b3def32b3b97671e60"
server_key_version: "encryptionkey@0"
I20260805 23:19:44.456660 13561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "dc76795b414c45f28df99d95d5049a70"
format_stamp: "Formatted at 2026-08-05 23:19:44 on dist-test-slave-vkjp"
server_key: "13b5a73fd04fbffc10a7c3bfb573994a"
server_key_iv: "763c52605b2807b3def32b3b97671e60"
server_key_version: "encryptionkey@0"
I20260805 23:19:44.460646 13561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:44.463308 13576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:44.464581 13561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:44.464735 13561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "dc76795b414c45f28df99d95d5049a70"
format_stamp: "Formatted at 2026-08-05 23:19:44 on dist-test-slave-vkjp"
server_key: "13b5a73fd04fbffc10a7c3bfb573994a"
server_key_iv: "763c52605b2807b3def32b3b97671e60"
server_key_version: "encryptionkey@0"
I20260805 23:19:44.464859 13561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:44.469570 13561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:44.472921 13561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:44.473160 13561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:44.481026 13561 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46191
I20260805 23:19:44.481057 13631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46191 every 8 connection(s)
I20260805 23:19:44.482048 13561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:44.484836 13632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:44.487743 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13561
I20260805 23:19:44.487926 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:44.488215 23905 external_mini_cluster.cc:1428] Setting key 399f8d15fa6595d63a8de9959f59b360
I20260805 23:19:44.491209 13632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70: Bootstrap starting.
I20260805 23:19:44.493464 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:44.489663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50881 (local address 127.0.0.1:46191)
0805 23:19:44.490131 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:44.490145 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:44.490179 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:19:44.490797 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:44.490812 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:44.491146 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:44.491530 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:44.492311 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:44.492315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:44.492325 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:44.492335 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:44.492388 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:44.492408 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:44.492557 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:44.492626 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 23:19:44.492778 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":147,"threads_started":1}
I20260805 23:19:44.493702 13632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:44.494630 13632 log.cc:824] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:44.496351 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:44.497021 13632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70: No bootstrap required, opened a new log
I20260805 23:19:44.500211 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:44.498402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55179 (local address 127.0.0.1:46191)
0805 23:19:44.498540 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:44.498546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:44.498770 (+ 224us) server_negotiation.cc:407] Connection header received
0805 23:19:44.498928 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:44.498931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:44.498978 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:44.499059 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:44.499802 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:44.499806 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:44.499808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:44.499820 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:44.499847 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:44.499866 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:44.499962 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:44.500040 (+ 78us) server_negotiation.cc:299] Negotiation successful
0805 23:19:44.500081 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:19:44.501394 13632 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260805 23:19:44.501616 13632 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:44.501688 13632 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc76795b414c45f28df99d95d5049a70, State: Initialized, Role: FOLLOWER
I20260805 23:19:44.502120 13632 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260805 23:19:44.502267 13632 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:44.502353 13632 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:44.502476 13632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:44.503619 13632 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260805 23:19:44.503968 13632 leader_election.cc:302] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc76795b414c45f28df99d95d5049a70; no voters:
I20260805 23:19:44.504289 13632 leader_election.cc:288] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:44.504426 13643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:44.504688 13643 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [term 1 LEADER]: Becoming Leader. State: Replica: dc76795b414c45f28df99d95d5049a70, State: Running, Role: LEADER
I20260805 23:19:44.505048 13643 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } }
I20260805 23:19:44.505595 13632 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:44.506891 13645 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc76795b414c45f28df99d95d5049a70. Latest consensus state: current_term: 1 leader_uuid: "dc76795b414c45f28df99d95d5049a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } } }
I20260805 23:19:44.507030 13645 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:44.507344 13644 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc76795b414c45f28df99d95d5049a70" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc76795b414c45f28df99d95d5049a70" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46191 } } }
I20260805 23:19:44.507488 13652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:44.508145 13644 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:44.510600 13652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:44.515552 13652 catalog_manager.cc:1347] Generated new cluster ID: 25ecd7f5b6f048c98b57530abe434d91
I20260805 23:19:44.515618 13652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:44.533063 13652 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:44.533397 13652 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:44.544272 13652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc76795b414c45f28df99d95d5049a70: Generated new TSK 0
I20260805 23:19:44.545012 13652 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:44.552264 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13561
2026-08-05T23:19:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:44 dist-test-slave-vkjp krb5kdc[13665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:44 dist-test-slave-vkjp krb5kdc[13665](info): set up 2 sockets
Aug 05 23:19:44 dist-test-slave-vkjp krb5kdc[13665](info): commencing operation
krb5kdc: starting...
W20260805 23:19:45.618432 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:45 dist-test-slave-vkjp krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:45.747849 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36707
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:45.858639 13679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:45.858898 13679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:45.858940 13679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:45.862834 13679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:45.862895 13679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:45.862914 13679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:45.862931 13679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:45.862946 13679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:45.868423 13679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36707
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:45.869410 13679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:45.870368 13679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:45.879195 13684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:45.879247 13679 server_base.cc:1034] running on GCE node
W20260805 23:19:45.879204 13687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:45.879540 13685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:45.880157 13679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:45.881323 13679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:45.882504 13679 hybrid_clock.cc:648] HybridClock initialized: now 1785971985882482 us; error 31 us; skew 500 ppm
Aug 05 23:19:45 dist-test-slave-vkjp krb5kdc[13665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971985, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:45.885555 13679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:45.886428 13679 webserver.cc:460] Webserver started at http://127.0.0.1:38775/ using document root <none> and password file <none>
I20260805 23:19:45.886962 13679 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:45.887024 13679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:45.887265 13679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:45.889408 13679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "663c4c4c343a4468a1c9a2cd89f88df4"
format_stamp: "Formatted at 2026-08-05 23:19:45 on dist-test-slave-vkjp"
server_key: "557524ad2f4168a0586bdbf1abe23c85"
server_key_iv: "cc9ca49140669e53aece2128b08fdaa3"
server_key_version: "encryptionkey@0"
I20260805 23:19:45.889958 13679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "663c4c4c343a4468a1c9a2cd89f88df4"
format_stamp: "Formatted at 2026-08-05 23:19:45 on dist-test-slave-vkjp"
server_key: "557524ad2f4168a0586bdbf1abe23c85"
server_key_iv: "cc9ca49140669e53aece2128b08fdaa3"
server_key_version: "encryptionkey@0"
I20260805 23:19:45.893745 13679 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:19:45.896215 13694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:45.897581 13679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260805 23:19:45.897729 13679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "663c4c4c343a4468a1c9a2cd89f88df4"
format_stamp: "Formatted at 2026-08-05 23:19:45 on dist-test-slave-vkjp"
server_key: "557524ad2f4168a0586bdbf1abe23c85"
server_key_iv: "cc9ca49140669e53aece2128b08fdaa3"
server_key_version: "encryptionkey@0"
I20260805 23:19:45.897853 13679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:45.918759 13679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:45.922493 13679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:45.922725 13679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:45.930728 13679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36707
I20260805 23:19:45.930754 13749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36707 every 8 connection(s)
I20260805 23:19:45.931741 13679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:45.934437 13750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:45.934854 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13679
I20260805 23:19:45.935007 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:45.935359 23905 external_mini_cluster.cc:1428] Setting key 7f5f0e87056b428a7241f1db81c816af
I20260805 23:19:45.941291 13750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4: Bootstrap starting.
I20260805 23:19:45.943536 13750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4: Neither blocks nor log segments found. Creating new log.
Aug 05 23:19:45 dist-test-slave-vkjp krb5kdc[13665](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:19:45.944487 13750 log.cc:824] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:45.946645 13750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4: No bootstrap required, opened a new log
I20260805 23:19:45.949801 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:45.936844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:54721 (local address 127.0.0.1:36707)
0805 23:19:45.937266 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:45.937280 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:45.937315 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:19:45.938009 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:45.938026 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:45.938366 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:45.938696 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:45.939575 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:45.940558 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:45.941491 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:45.941956 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:45.942129 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:45.944987 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:45.944992 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:45.945004 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:45.945032 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:45.946892 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:45.947456 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:45.947462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:45.947469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:45.947591 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:45.947969 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:45.947971 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:45.947972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:45.948333 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:45.948573 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:45.948890 (+ 317us) server_negotiation.cc:299] Negotiation successful
0805 23:19:45.949105 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260805 23:19:45.950382 13750 raft_consensus.cc:348] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } }
I20260805 23:19:45.950654 13750 raft_consensus.cc:374] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:45.950749 13750 raft_consensus.cc:729] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 663c4c4c343a4468a1c9a2cd89f88df4, State: Initialized, Role: FOLLOWER
I20260805 23:19:45.951273 13750 consensus_queue.cc:260] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } }
I20260805 23:19:45.951499 13750 raft_consensus.cc:388] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:45.951579 13750 raft_consensus.cc:482] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:45.951689 13750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:45.952913 13750 raft_consensus.cc:504] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } }
I20260805 23:19:45.953341 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:45.953301 13750 leader_election.cc:302] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 663c4c4c343a4468a1c9a2cd89f88df4; no voters:
I20260805 23:19:45.953672 13750 leader_election.cc:288] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:45.953862 13755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:45.954159 13755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [term 1 LEADER]: Becoming Leader. State: Replica: 663c4c4c343a4468a1c9a2cd89f88df4, State: Running, Role: LEADER
I20260805 23:19:45.954545 13755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } }
I20260805 23:19:45.954910 13750 sys_catalog.cc:564] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:45.955940 13755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 663c4c4c343a4468a1c9a2cd89f88df4. Latest consensus state: current_term: 1 leader_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } } }
I20260805 23:19:45.956112 13755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:45.956372 13756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "663c4c4c343a4468a1c9a2cd89f88df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36707 } } }
I20260805 23:19:45.956475 13756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:45.956491 13768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:45.959688 13768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:45.965569 13768 catalog_manager.cc:1347] Generated new cluster ID: 7aa060f3af3743f193fc2475f7533a29
I20260805 23:19:45.965633 13768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:45.967244 13753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:45.955687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:40467 (local address 127.0.0.1:36707)
0805 23:19:45.955783 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:45.955790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:45.957037 (+ 1247us) server_negotiation.cc:407] Connection header received
0805 23:19:45.957249 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:45.957256 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:45.957329 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:45.957388 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:45.958363 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:45.959116 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:45.960417 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:45.960853 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:45.960938 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:45.962965 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:45.962969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:45.962972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:45.962999 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:45.965087 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:45.965855 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:45.965858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:45.965860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:45.965949 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:45.966301 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:45.966305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:45.966306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:45.966509 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:45.966683 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:45.966954 (+ 271us) server_negotiation.cc:299] Negotiation successful
0805 23:19:45.967053 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":25}
I20260805 23:19:45.979671 13768 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:45.980788 13768 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:45.986349 13768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 663c4c4c343a4468a1c9a2cd89f88df4: Generated new TSK 0
I20260805 23:19:45.987028 13768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:45.992815 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13679
W20260805 23:19:45.999784 13663 connection.cc:535] client connection to 127.0.0.1:36707 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36707): Connection reset by peer (error 104)
2026-08-05T23:19:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:46 dist-test-slave-vkjp krb5kdc[13781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:46 dist-test-slave-vkjp krb5kdc[13781](info): set up 2 sockets
Aug 05 23:19:46 dist-test-slave-vkjp krb5kdc[13781](info): commencing operation
krb5kdc: starting...
W20260805 23:19:47.108573 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:47 dist-test-slave-vkjp krb5kdc[13781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:47.234583 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45345
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:47.344039 13794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:47.344337 13794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:47.344422 13794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:47.348362 13794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:47.348454 13794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:47.348495 13794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:47.348548 13794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:47.348587 13794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:47.354012 13794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:47.355065 13794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:47.356087 13794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:47.364580 13800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:47.364603 13799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:47.364533 13802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:47.364817 13794 server_base.cc:1034] running on GCE node
I20260805 23:19:47.365411 13794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:47.366433 13794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:47.367601 13794 hybrid_clock.cc:648] HybridClock initialized: now 1785971987367584 us; error 28 us; skew 500 ppm
Aug 05 23:19:47 dist-test-slave-vkjp krb5kdc[13781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971987, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:47.370265 13794 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:47.371263 13794 webserver.cc:460] Webserver started at http://127.0.0.1:36265/ using document root <none> and password file <none>
I20260805 23:19:47.371809 13794 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:47.371869 13794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:47.372093 13794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:47.374118 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "f97527ab72b342a58636577d5bb7b73b"
format_stamp: "Formatted at 2026-08-05 23:19:47 on dist-test-slave-vkjp"
server_key: "0927ab9c94128e85f5a1d62a8d7b4892"
server_key_iv: "f43e07297b74558df99fbf43b8167b8f"
server_key_version: "encryptionkey@0"
I20260805 23:19:47.374678 13794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "f97527ab72b342a58636577d5bb7b73b"
format_stamp: "Formatted at 2026-08-05 23:19:47 on dist-test-slave-vkjp"
server_key: "0927ab9c94128e85f5a1d62a8d7b4892"
server_key_iv: "f43e07297b74558df99fbf43b8167b8f"
server_key_version: "encryptionkey@0"
I20260805 23:19:47.378448 13794 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260805 23:19:47.380926 13809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:47.382279 13794 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260805 23:19:47.382427 13794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "f97527ab72b342a58636577d5bb7b73b"
format_stamp: "Formatted at 2026-08-05 23:19:47 on dist-test-slave-vkjp"
server_key: "0927ab9c94128e85f5a1d62a8d7b4892"
server_key_iv: "f43e07297b74558df99fbf43b8167b8f"
server_key_version: "encryptionkey@0"
I20260805 23:19:47.382547 13794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:47.407430 13794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:47.410382 13794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:47.410640 13794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:47.418058 13794 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45345
I20260805 23:19:47.418085 13865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45345 every 8 connection(s)
I20260805 23:19:47.419030 13794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:47.420964 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13794
I20260805 23:19:47.421118 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:47.421394 23905 external_mini_cluster.cc:1428] Setting key 230d81b6be38a4afdf8bfc00a75162b8
I20260805 23:19:47.421952 13866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:47.428952 13866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b: Bootstrap starting.
I20260805 23:19:47.431373 13869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:47.422809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:49515 (local address 127.0.0.1:45345)
0805 23:19:47.423285 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:47.423306 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:47.423350 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:19:47.424157 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:47.424180 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:47.424607 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:47.424973 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:47.425369 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:47.426817 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:47.427769 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:47.428444 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:47.428592 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:47.429443 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:47.429449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:47.429464 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:47.429935 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:47.430004 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:47.430039 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:47.430182 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:47.430331 (+ 149us) server_negotiation.cc:299] Negotiation successful
0805 23:19:47.430464 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":134,"threads_started":1}
I20260805 23:19:47.431638 13866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:47.432389 13866 log.cc:824] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:47.434219 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:47.434492 13866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b: No bootstrap required, opened a new log
I20260805 23:19:47.438064 13866 raft_consensus.cc:348] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } }
I20260805 23:19:47.438303 13866 raft_consensus.cc:374] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:47.438416 13866 raft_consensus.cc:729] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f97527ab72b342a58636577d5bb7b73b, State: Initialized, Role: FOLLOWER
I20260805 23:19:47.438889 13866 consensus_queue.cc:260] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } }
I20260805 23:19:47.439090 13866 raft_consensus.cc:388] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:47.439210 13866 raft_consensus.cc:482] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:47.439312 13866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:47.440209 13869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:47.436245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:36413 (local address 127.0.0.1:45345)
0805 23:19:47.436373 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:47.436378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:47.436572 (+ 194us) server_negotiation.cc:407] Connection header received
0805 23:19:47.436688 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:47.436691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:47.436732 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:47.436819 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:47.437083 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:47.437572 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:47.438371 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:47.438735 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:47.438833 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:47.439771 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:47.439778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:47.439781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:47.439794 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:47.439821 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:47.439834 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:47.439934 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:47.440039 (+ 105us) server_negotiation.cc:299] Negotiation successful
0805 23:19:47.440075 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:19:47.440536 13866 raft_consensus.cc:504] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } }
I20260805 23:19:47.440904 13866 leader_election.cc:302] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f97527ab72b342a58636577d5bb7b73b; no voters:
I20260805 23:19:47.441248 13866 leader_election.cc:288] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:47.441320 13876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:47.441504 13876 raft_consensus.cc:686] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [term 1 LEADER]: Becoming Leader. State: Replica: f97527ab72b342a58636577d5bb7b73b, State: Running, Role: LEADER
I20260805 23:19:47.441794 13876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } }
I20260805 23:19:47.442544 13866 sys_catalog.cc:564] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:47.443540 13878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f97527ab72b342a58636577d5bb7b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } } }
I20260805 23:19:47.443672 13878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:47.444012 13877 sys_catalog.cc:455] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f97527ab72b342a58636577d5bb7b73b. Latest consensus state: current_term: 1 leader_uuid: "f97527ab72b342a58636577d5bb7b73b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f97527ab72b342a58636577d5bb7b73b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45345 } } }
I20260805 23:19:47.444084 13886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:47.444130 13877 sys_catalog.cc:458] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:47.447353 13886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:47.452919 13886 catalog_manager.cc:1347] Generated new cluster ID: 6caa314dfae04123a2ac0dd80e17f332
I20260805 23:19:47.452996 13886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:47.459838 13886 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:47.461094 13886 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:47.474236 13886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f97527ab72b342a58636577d5bb7b73b: Generated new TSK 0
I20260805 23:19:47.475111 13886 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:47.489516 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13794
W20260805 23:19:47.496028 13779 connection.cc:535] client connection to 127.0.0.1:45345 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45345): Connection reset by peer (error 104)
2026-08-05T23:19:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:47 dist-test-slave-vkjp krb5kdc[13898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:47 dist-test-slave-vkjp krb5kdc[13898](info): set up 2 sockets
Aug 05 23:19:47 dist-test-slave-vkjp krb5kdc[13898](info): commencing operation
krb5kdc: starting...
W20260805 23:19:48.576610 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:48 dist-test-slave-vkjp krb5kdc[13898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:48.701848 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:48.813450 13911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:48.813763 13911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:48.813841 13911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:48.817981 13911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:48.818089 13911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:48.818127 13911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:48.818187 13911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:48.818226 13911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:48.823666 13911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42059
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:48.824827 13911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:48.825886 13911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:48.834617 13916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:48.834720 13911 server_base.cc:1034] running on GCE node
W20260805 23:19:48.834614 13919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:48.834664 13917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:48.835539 13911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:48.836606 13911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:48.837790 13911 hybrid_clock.cc:648] HybridClock initialized: now 1785971988837764 us; error 40 us; skew 500 ppm
Aug 05 23:19:48 dist-test-slave-vkjp krb5kdc[13898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:48.844501 13911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:48.845494 13911 webserver.cc:460] Webserver started at http://127.0.0.1:36439/ using document root <none> and password file <none>
I20260805 23:19:48.846086 13911 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:48.846182 13911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:48.846421 13911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:48.848595 13911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "f95ff90f7bd848d19f8fddfa66a933c7"
format_stamp: "Formatted at 2026-08-05 23:19:48 on dist-test-slave-vkjp"
server_key: "d6ca6236a03a66e9aebabe88b20ad4b1"
server_key_iv: "d4ac03e5096dbf3aa143aa93e0b54f2f"
server_key_version: "encryptionkey@0"
I20260805 23:19:48.849170 13911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "f95ff90f7bd848d19f8fddfa66a933c7"
format_stamp: "Formatted at 2026-08-05 23:19:48 on dist-test-slave-vkjp"
server_key: "d6ca6236a03a66e9aebabe88b20ad4b1"
server_key_iv: "d4ac03e5096dbf3aa143aa93e0b54f2f"
server_key_version: "encryptionkey@0"
I20260805 23:19:48.852900 13911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260805 23:19:48.855484 13926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:48.856678 13911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:48.856820 13911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "f95ff90f7bd848d19f8fddfa66a933c7"
format_stamp: "Formatted at 2026-08-05 23:19:48 on dist-test-slave-vkjp"
server_key: "d6ca6236a03a66e9aebabe88b20ad4b1"
server_key_iv: "d4ac03e5096dbf3aa143aa93e0b54f2f"
server_key_version: "encryptionkey@0"
I20260805 23:19:48.856935 13911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:48.861552 13911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:48.864813 13911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:48.865027 13911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:48.872388 13911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42059
I20260805 23:19:48.872413 13981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42059 every 8 connection(s)
I20260805 23:19:48.873351 13911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:48.876164 13982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:48.878527 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 13911
I20260805 23:19:48.878700 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:48.878990 23905 external_mini_cluster.cc:1428] Setting key fce0481c8a104cc3849094a29820fe9b
I20260805 23:19:48.882336 13982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7: Bootstrap starting.
I20260805 23:19:48.884442 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:48.880503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:42311 (local address 127.0.0.1:42059)
0805 23:19:48.881003 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:48.881018 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:48.881052 (+ 34us) server_negotiation.cc:407] Connection header received
0805 23:19:48.881692 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:48.881709 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:48.881996 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:48.882335 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:48.883146 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:48.883152 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:48.883166 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:48.883185 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:48.883243 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:48.883263 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:48.883415 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:48.883484 (+ 69us) server_negotiation.cc:299] Negotiation successful
0805 23:19:48.883759 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":162,"threads_started":1}
I20260805 23:19:48.884820 13982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:48.885651 13982 log.cc:824] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:48.887403 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:48.887925 13982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7: No bootstrap required, opened a new log
I20260805 23:19:48.891367 13985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:48.889710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:43701 (local address 127.0.0.1:42059)
0805 23:19:48.889826 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:48.889832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:48.889983 (+ 151us) server_negotiation.cc:407] Connection header received
0805 23:19:48.890089 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:48.890092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:48.890127 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:48.890215 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:48.890930 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:48.890934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:48.890939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:48.890952 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:48.890983 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:48.890995 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:48.891106 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:48.891200 (+ 94us) server_negotiation.cc:299] Negotiation successful
0805 23:19:48.891247 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260805 23:19:48.892702 13982 raft_consensus.cc:348] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } }
I20260805 23:19:48.892915 13982 raft_consensus.cc:374] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:48.892992 13982 raft_consensus.cc:729] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f95ff90f7bd848d19f8fddfa66a933c7, State: Initialized, Role: FOLLOWER
I20260805 23:19:48.893391 13982 consensus_queue.cc:260] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } }
I20260805 23:19:48.893536 13982 raft_consensus.cc:388] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:48.893605 13982 raft_consensus.cc:482] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:48.893702 13982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:48.894635 13982 raft_consensus.cc:504] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } }
I20260805 23:19:48.894979 13982 leader_election.cc:302] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f95ff90f7bd848d19f8fddfa66a933c7; no voters:
I20260805 23:19:48.895309 13982 leader_election.cc:288] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:48.895424 13992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:48.895648 13992 raft_consensus.cc:686] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [term 1 LEADER]: Becoming Leader. State: Replica: f95ff90f7bd848d19f8fddfa66a933c7, State: Running, Role: LEADER
I20260805 23:19:48.896000 13992 consensus_queue.cc:237] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } }
I20260805 23:19:48.896632 13982 sys_catalog.cc:564] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:48.897123 13992 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f95ff90f7bd848d19f8fddfa66a933c7. Latest consensus state: current_term: 1 leader_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } } }
I20260805 23:19:48.897291 13992 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:48.897764 13993 sys_catalog.cc:455] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f95ff90f7bd848d19f8fddfa66a933c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42059 } } }
I20260805 23:19:48.897864 13993 sys_catalog.cc:458] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:48.897894 13998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:48.901062 13998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:48.906837 13998 catalog_manager.cc:1347] Generated new cluster ID: 47d43e8e35004627b13da6ec3358c85a
I20260805 23:19:48.906906 13998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:48.936605 13998 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:48.936993 13998 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:48.941092 13998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f95ff90f7bd848d19f8fddfa66a933c7: Generated new TSK 0
I20260805 23:19:48.941756 13998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:48.945039 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 13911
2026-08-05T23:19:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:49 dist-test-slave-vkjp krb5kdc[14014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:49 dist-test-slave-vkjp krb5kdc[14014](info): set up 2 sockets
Aug 05 23:19:49 dist-test-slave-vkjp krb5kdc[14014](info): commencing operation
krb5kdc: starting...
W20260805 23:19:50.028903 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:50.155164 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:50.268980 14029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:50.269276 14029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:50.269320 14029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:50.273306 14029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:50.273365 14029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:50.273386 14029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:50.273402 14029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:50.273417 14029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:50.278724 14029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:50.279793 14029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:50.280776 14029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:50.289227 14034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:50.289235 14037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:50.289235 14035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:50.289413 14029 server_base.cc:1034] running on GCE node
I20260805 23:19:50.290126 14029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:50.291316 14029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:50.292493 14029 hybrid_clock.cc:648] HybridClock initialized: now 1785971990292475 us; error 29 us; skew 500 ppm
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971990, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:50.295341 14029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:50.296277 14029 webserver.cc:460] Webserver started at http://127.0.0.1:34553/ using document root <none> and password file <none>
I20260805 23:19:50.296802 14029 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:50.296864 14029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:50.297035 14029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:50.299052 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "4b5ef695afda494da03e334087d97277"
format_stamp: "Formatted at 2026-08-05 23:19:50 on dist-test-slave-vkjp"
server_key: "c76d909d52673d6f99f1b0ebc109c070"
server_key_iv: "c0e6f2af70a9c1e80d1c73bf41ca723b"
server_key_version: "encryptionkey@0"
I20260805 23:19:50.299651 14029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "4b5ef695afda494da03e334087d97277"
format_stamp: "Formatted at 2026-08-05 23:19:50 on dist-test-slave-vkjp"
server_key: "c76d909d52673d6f99f1b0ebc109c070"
server_key_iv: "c0e6f2af70a9c1e80d1c73bf41ca723b"
server_key_version: "encryptionkey@0"
I20260805 23:19:50.303550 14029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:19:50.305925 14044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:50.307428 14029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:50.307564 14029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "4b5ef695afda494da03e334087d97277"
format_stamp: "Formatted at 2026-08-05 23:19:50 on dist-test-slave-vkjp"
server_key: "c76d909d52673d6f99f1b0ebc109c070"
server_key_iv: "c0e6f2af70a9c1e80d1c73bf41ca723b"
server_key_version: "encryptionkey@0"
I20260805 23:19:50.307673 14029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:50.320216 14029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:50.323571 14029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:50.323789 14029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:50.331149 14029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43403
I20260805 23:19:50.331200 14097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43403 every 8 connection(s)
I20260805 23:19:50.332124 14029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:50.334830 14098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:50.341081 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277: Bootstrap starting.
I20260805 23:19:50.342128 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14029
I20260805 23:19:50.342278 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:50.342631 23905 external_mini_cluster.cc:1428] Setting key ed47bab7784d1745b3db9ac1eb23ea5a
I20260805 23:19:50.344182 14098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:50.344925 14098 log.cc:824] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:50.346879 14098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277: No bootstrap required, opened a new log
I20260805 23:19:50.349932 14098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } }
I20260805 23:19:50.350180 14098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:50.350270 14098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5ef695afda494da03e334087d97277, State: Initialized, Role: FOLLOWER
I20260805 23:19:50.350726 14098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } }
I20260805 23:19:50.350880 14098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:50.350970 14098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:50.351112 14098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:50.352039 14098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } }
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260805 23:19:50.352422 14098 leader_election.cc:302] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b5ef695afda494da03e334087d97277; no voters:
I20260805 23:19:50.352768 14098 leader_election.cc:288] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:50.352841 14103 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:50.353053 14103 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [term 1 LEADER]: Becoming Leader. State: Replica: 4b5ef695afda494da03e334087d97277, State: Running, Role: LEADER
I20260805 23:19:50.353406 14103 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } }
I20260805 23:19:50.354115 14098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:50.355165 14105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b5ef695afda494da03e334087d97277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } } }
I20260805 23:19:50.355297 14105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:50.356515 14104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b5ef695afda494da03e334087d97277. Latest consensus state: current_term: 1 leader_uuid: "4b5ef695afda494da03e334087d97277" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5ef695afda494da03e334087d97277" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43403 } } }
I20260805 23:19:50.356719 14104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:50.357017 14113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:50.359345 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:50.343836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:58661 (local address 127.0.0.1:43403)
0805 23:19:50.344330 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:50.344347 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:50.344384 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:19:50.345104 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:50.345123 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:50.345546 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:50.345947 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:50.346775 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:50.347886 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:50.348754 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:50.349801 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:50.349990 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:50.352959 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:50.352966 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:50.352983 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:50.353021 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:50.355254 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:50.356055 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:50.356064 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:50.356074 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:50.356227 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:50.356666 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:50.356670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:50.356672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:50.357211 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:50.357463 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:50.358074 (+ 611us) server_negotiation.cc:299] Negotiation successful
0805 23:19:50.358314 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":141,"threads_started":1}
I20260805 23:19:50.360082 14113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:50.362632 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:50.366577 14113 catalog_manager.cc:1347] Generated new cluster ID: f5f9fa79c05844268d922a96980d758c
I20260805 23:19:50.366647 14113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:50.375487 14101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:50.364998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:54479 (local address 127.0.0.1:43403)
0805 23:19:50.365107 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:50.365112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:50.366595 (+ 1483us) server_negotiation.cc:407] Connection header received
0805 23:19:50.366793 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:50.366799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:50.366855 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:50.366952 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:50.367812 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:50.368645 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:50.369748 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:50.370138 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:50.370248 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:50.371889 (+ 1641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:50.371893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:50.371896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:50.371924 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:50.373607 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:50.374115 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:50.374119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:50.374122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:50.374211 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:50.374549 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:50.374552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:50.374554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:50.374764 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:50.374914 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:50.375215 (+ 301us) server_negotiation.cc:299] Negotiation successful
0805 23:19:50.375318 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260805 23:19:50.379565 14113 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:50.380621 14113 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:50.392866 14113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b5ef695afda494da03e334087d97277: Generated new TSK 0
I20260805 23:19:50.393657 14113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:50.398130 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14029
W20260805 23:19:50.405773 14011 connection.cc:535] client connection to 127.0.0.1:43403 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43403): Connection reset by peer (error 104)
2026-08-05T23:19:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14131](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14131](info): set up 2 sockets
Aug 05 23:19:50 dist-test-slave-vkjp krb5kdc[14131](info): commencing operation
krb5kdc: starting...
W20260805 23:19:51.464737 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:51 dist-test-slave-vkjp krb5kdc[14131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:51.591265 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45243
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:51.706315 14145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:51.706602 14145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:51.706646 14145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:51.710772 14145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:51.710837 14145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:51.710857 14145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:51.710875 14145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:51.710891 14145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:51.716149 14145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45243
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:51.717198 14145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:51.718255 14145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:51.726640 14153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:51.726830 14145 server_base.cc:1034] running on GCE node
W20260805 23:19:51.726657 14151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:51.726647 14150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:51.727591 14145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:51.728778 14145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:51.729969 14145 hybrid_clock.cc:648] HybridClock initialized: now 1785971991729936 us; error 48 us; skew 500 ppm
Aug 05 23:19:51 dist-test-slave-vkjp krb5kdc[14131](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:51.732975 14145 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:51.733946 14145 webserver.cc:460] Webserver started at http://127.0.0.1:34643/ using document root <none> and password file <none>
I20260805 23:19:51.734524 14145 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:51.734586 14145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:51.734824 14145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:51.737040 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "22fa11c3335e4c93bf45365bc34f7db3"
format_stamp: "Formatted at 2026-08-05 23:19:51 on dist-test-slave-vkjp"
server_key: "b4ae3cc5c154754b393293e1d9c867a1"
server_key_iv: "c92a31502c741061fa441044b25f5f72"
server_key_version: "encryptionkey@0"
I20260805 23:19:51.737617 14145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "22fa11c3335e4c93bf45365bc34f7db3"
format_stamp: "Formatted at 2026-08-05 23:19:51 on dist-test-slave-vkjp"
server_key: "b4ae3cc5c154754b393293e1d9c867a1"
server_key_iv: "c92a31502c741061fa441044b25f5f72"
server_key_version: "encryptionkey@0"
I20260805 23:19:51.741333 14145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260805 23:19:51.743777 14161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:51.744992 14145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260805 23:19:51.745127 14145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "22fa11c3335e4c93bf45365bc34f7db3"
format_stamp: "Formatted at 2026-08-05 23:19:51 on dist-test-slave-vkjp"
server_key: "b4ae3cc5c154754b393293e1d9c867a1"
server_key_iv: "c92a31502c741061fa441044b25f5f72"
server_key_version: "encryptionkey@0"
I20260805 23:19:51.745240 14145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:51.776031 14145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:51.780085 14145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:51.780324 14145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:51.790838 14214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45243 every 8 connection(s)
I20260805 23:19:51.790845 14145 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45243
I20260805 23:19:51.792326 14145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:51.795987 14215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:51.799974 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14145
I20260805 23:19:51.800123 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:51.800433 23905 external_mini_cluster.cc:1428] Setting key 9e8416efeb7e5f611318b9cbf3e24d8b
I20260805 23:19:51.802863 14215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3: Bootstrap starting.
I20260805 23:19:51.805193 14215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:51.805922 14215 log.cc:824] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:51.807879 14215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3: No bootstrap required, opened a new log
I20260805 23:19:51.809352 14218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:51.801828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:36373 (local address 127.0.0.1:45243)
0805 23:19:51.802267 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:51.802289 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:51.802326 (+ 37us) server_negotiation.cc:407] Connection header received
0805 23:19:51.802996 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:51.803023 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:51.803412 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:51.803722 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:51.804141 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:51.805122 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:51.806067 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:51.806539 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:51.806704 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:51.807654 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:51.807659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:51.807671 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:51.808107 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:51.808161 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:51.808184 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:51.808326 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:51.808520 (+ 194us) server_negotiation.cc:299] Negotiation successful
0805 23:19:51.808656 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260805 23:19:51.811883 14215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } }
I20260805 23:19:51.812157 14215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:51.812222 14215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22fa11c3335e4c93bf45365bc34f7db3, State: Initialized, Role: FOLLOWER
I20260805 23:19:51.812533 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:51.812691 14215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } }
I20260805 23:19:51.812839 14215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:51.812899 14215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:51.812994 14215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:51.814258 14215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } }
I20260805 23:19:51.814642 14215 leader_election.cc:302] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22fa11c3335e4c93bf45365bc34f7db3; no voters:
I20260805 23:19:51.814989 14215 leader_election.cc:288] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:51.815052 14224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:51.815305 14224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [term 1 LEADER]: Becoming Leader. State: Replica: 22fa11c3335e4c93bf45365bc34f7db3, State: Running, Role: LEADER
I20260805 23:19:51.815682 14224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } }
I20260805 23:19:51.816310 14215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:51.816915 14224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22fa11c3335e4c93bf45365bc34f7db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } } }
I20260805 23:19:51.817081 14224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:51.817340 14226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22fa11c3335e4c93bf45365bc34f7db3. Latest consensus state: current_term: 1 leader_uuid: "22fa11c3335e4c93bf45365bc34f7db3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22fa11c3335e4c93bf45365bc34f7db3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45243 } } }
I20260805 23:19:51.817478 14226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:51.817498 14234 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:51.819722 14218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:51.814907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:46571 (local address 127.0.0.1:45243)
0805 23:19:51.815036 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:51.815041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:51.815377 (+ 336us) server_negotiation.cc:407] Connection header received
0805 23:19:51.815605 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:51.815608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:51.815659 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:51.815743 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:51.816020 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:51.816594 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:51.817839 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:51.818231 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:51.818326 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:51.819230 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:51.819233 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:51.819236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:51.819248 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:51.819277 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:51.819292 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:51.819370 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:51.819533 (+ 163us) server_negotiation.cc:299] Negotiation successful
0805 23:19:51.819582 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:19:51.821472 14234 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:51.827229 14234 catalog_manager.cc:1347] Generated new cluster ID: 8f168829b0974473acd2c0505b6b21b6
I20260805 23:19:51.827312 14234 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:51.849020 14234 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:51.850057 14234 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:51.856142 14234 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22fa11c3335e4c93bf45365bc34f7db3: Generated new TSK 0
I20260805 23:19:51.856835 14234 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:51.867697 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14145
W20260805 23:19:51.874611 14129 connection.cc:535] client connection to 127.0.0.1:45243 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45243): Connection reset by peer (error 104)
2026-08-05T23:19:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1469 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:51 dist-test-slave-vkjp krb5kdc[14249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:51 dist-test-slave-vkjp krb5kdc[14249](info): set up 2 sockets
Aug 05 23:19:51 dist-test-slave-vkjp krb5kdc[14249](info): commencing operation
krb5kdc: starting...
W20260805 23:19:52.937134 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:53 dist-test-slave-vkjp krb5kdc[14249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260805 23:19:53.064556 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40609
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40609
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:53.177260 14262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:53.177534 14262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:53.177709 14262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:53.181897 14262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:53.181968 14262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:53.181988 14262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:53.182004 14262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:53.182052 14262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:53.187314 14262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40609
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:53.188395 14262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:53.189461 14262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:53.197850 14270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:53.197962 14268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:53.197925 14267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:53.198060 14262 server_base.cc:1034] running on GCE node
I20260805 23:19:53.198722 14262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:53.199915 14262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:53.201102 14262 hybrid_clock.cc:648] HybridClock initialized: now 1785971993201079 us; error 36 us; skew 500 ppm
Aug 05 23:19:53 dist-test-slave-vkjp krb5kdc[14249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971993, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:53.203914 14262 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260805 23:19:53.204761 14262 webserver.cc:460] Webserver started at http://127.0.0.1:41669/ using document root <none> and password file <none>
I20260805 23:19:53.205302 14262 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:53.205389 14262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:53.205641 14262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:53.207671 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "a35d0a60299c4ad99f7be7cfc47de3cb"
format_stamp: "Formatted at 2026-08-05 23:19:53 on dist-test-slave-vkjp"
server_key: "30eb7148f3105684651e9696048e4eba"
server_key_iv: "ae182e7f46c3a65a1b787f18c0fee31a"
server_key_version: "encryptionkey@0"
I20260805 23:19:53.208230 14262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "a35d0a60299c4ad99f7be7cfc47de3cb"
format_stamp: "Formatted at 2026-08-05 23:19:53 on dist-test-slave-vkjp"
server_key: "30eb7148f3105684651e9696048e4eba"
server_key_iv: "ae182e7f46c3a65a1b787f18c0fee31a"
server_key_version: "encryptionkey@0"
I20260805 23:19:53.211930 14262 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260805 23:19:53.214340 14277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:53.215617 14262 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:53.215757 14262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "a35d0a60299c4ad99f7be7cfc47de3cb"
format_stamp: "Formatted at 2026-08-05 23:19:53 on dist-test-slave-vkjp"
server_key: "30eb7148f3105684651e9696048e4eba"
server_key_iv: "ae182e7f46c3a65a1b787f18c0fee31a"
server_key_version: "encryptionkey@0"
I20260805 23:19:53.215867 14262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:53.220409 14262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:53.223737 14262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:53.223937 14262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:53.230858 14262 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40609
I20260805 23:19:53.230979 14333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40609 every 8 connection(s)
I20260805 23:19:53.231829 14262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:53.234560 14334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:53.240027 14334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb: Bootstrap starting.
I20260805 23:19:53.240558 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14262
I20260805 23:19:53.240725 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:53.241019 23905 external_mini_cluster.cc:1428] Setting key 1ac15b62d93a7cae4f34bcbc2ea46490
I20260805 23:19:53.242830 14334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:53.243733 14334 log.cc:824] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:53.245853 14334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb: No bootstrap required, opened a new log
I20260805 23:19:53.246517 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:53.242363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:47509 (local address 127.0.0.1:40609)
0805 23:19:53.242829 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:53.242853 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:53.242908 (+ 55us) server_negotiation.cc:407] Connection header received
0805 23:19:53.243799 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:53.243824 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:53.244203 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:53.244546 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:53.245316 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:53.245320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:53.245330 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:53.245341 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:53.245392 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:53.245413 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:53.245524 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:53.245624 (+ 100us) server_negotiation.cc:299] Negotiation successful
0805 23:19:53.245797 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":127,"threads_started":1}
I20260805 23:19:53.249174 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:53.250460 14334 raft_consensus.cc:348] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } }
I20260805 23:19:53.250701 14334 raft_consensus.cc:374] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:53.250805 14334 raft_consensus.cc:729] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a35d0a60299c4ad99f7be7cfc47de3cb, State: Initialized, Role: FOLLOWER
I20260805 23:19:53.251325 14334 consensus_queue.cc:260] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } }
I20260805 23:19:53.251549 14334 raft_consensus.cc:388] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:53.251650 14334 raft_consensus.cc:482] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:53.251760 14334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:53.252972 14337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:53.251335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:57611 (local address 127.0.0.1:40609)
0805 23:19:53.251480 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:53.251484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:53.251627 (+ 143us) server_negotiation.cc:407] Connection header received
0805 23:19:53.251759 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:53.251762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:53.251793 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:53.251882 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:53.252571 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:53.252574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:53.252576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:53.252588 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:53.252614 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:53.252632 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:53.252722 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:53.252814 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 23:19:53.252855 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:19:53.252920 14334 raft_consensus.cc:504] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } }
I20260805 23:19:53.253315 14334 leader_election.cc:302] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a35d0a60299c4ad99f7be7cfc47de3cb; no voters:
I20260805 23:19:53.253657 14334 leader_election.cc:288] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:53.253752 14344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:53.254017 14344 raft_consensus.cc:686] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [term 1 LEADER]: Becoming Leader. State: Replica: a35d0a60299c4ad99f7be7cfc47de3cb, State: Running, Role: LEADER
I20260805 23:19:53.254348 14344 consensus_queue.cc:237] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } }
I20260805 23:19:53.254881 14334 sys_catalog.cc:564] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:53.255543 14346 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a35d0a60299c4ad99f7be7cfc47de3cb. Latest consensus state: current_term: 1 leader_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } } }
I20260805 23:19:53.255668 14346 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:53.255960 14345 sys_catalog.cc:455] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a35d0a60299c4ad99f7be7cfc47de3cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40609 } } }
I20260805 23:19:53.256062 14345 sys_catalog.cc:458] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:53.256134 14351 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:53.259251 14351 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:53.264688 14351 catalog_manager.cc:1347] Generated new cluster ID: 0ad42ca62f1d4f66bf57ca643d3d6be8
I20260805 23:19:53.264745 14351 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:53.294308 14351 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:53.294657 14351 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:53.305126 14351 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a35d0a60299c4ad99f7be7cfc47de3cb: Generated new TSK 0
I20260805 23:19:53.305927 14351 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:53.310322 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14262
2026-08-05T23:19:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:53 dist-test-slave-vkjp krb5kdc[14365](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:53 dist-test-slave-vkjp krb5kdc[14365](info): set up 2 sockets
Aug 05 23:19:53 dist-test-slave-vkjp krb5kdc[14365](info): commencing operation
krb5kdc: starting...
W20260805 23:19:54.402544 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:54.528007 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44877
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:37735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:44877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:54.641439 14378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:54.641711 14378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:54.641755 14378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:54.646020 14378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:54.646093 14378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:54.646113 14378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:54.646131 14378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:54.646147 14378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:54.651518 14378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:37735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:44877
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:44877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:54.652599 14378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:54.653666 14378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:54.662619 14383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:54.662690 14378 server_base.cc:1034] running on GCE node
W20260805 23:19:54.662616 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:54.662597 14384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:54.663509 14378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:54.664685 14378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:54.665894 14378 hybrid_clock.cc:648] HybridClock initialized: now 1785971994665873 us; error 33 us; skew 500 ppm
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14365](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971994, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:54.672370 14378 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:54.673350 14378 webserver.cc:460] Webserver started at http://127.23.88.126:41605/ using document root <none> and password file <none>
I20260805 23:19:54.673895 14378 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:54.673959 14378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:54.674127 14378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:54.676296 14378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "9017ee1cac994b259cc8d643fa8f2227"
format_stamp: "Formatted at 2026-08-05 23:19:54 on dist-test-slave-vkjp"
server_key: "8ced6e7cbfa19e3fd80a6f445c7383af"
server_key_iv: "699dc75140c77ac4f698864cef2372cb"
server_key_version: "encryptionkey@0"
I20260805 23:19:54.676867 14378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "9017ee1cac994b259cc8d643fa8f2227"
format_stamp: "Formatted at 2026-08-05 23:19:54 on dist-test-slave-vkjp"
server_key: "8ced6e7cbfa19e3fd80a6f445c7383af"
server_key_iv: "699dc75140c77ac4f698864cef2372cb"
server_key_version: "encryptionkey@0"
I20260805 23:19:54.680716 14378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:54.683210 14394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:54.684368 14378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:54.684520 14378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "9017ee1cac994b259cc8d643fa8f2227"
format_stamp: "Formatted at 2026-08-05 23:19:54 on dist-test-slave-vkjp"
server_key: "8ced6e7cbfa19e3fd80a6f445c7383af"
server_key_iv: "699dc75140c77ac4f698864cef2372cb"
server_key_version: "encryptionkey@0"
I20260805 23:19:54.684617 14378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:54.701179 14378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:54.704313 14378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:54.704535 14378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:54.711930 14378 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:44877
I20260805 23:19:54.711956 14448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:44877 every 8 connection(s)
I20260805 23:19:54.712867 14378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:54.715008 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14378
I20260805 23:19:54.715184 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:54.715494 23905 external_mini_cluster.cc:1428] Setting key a6c74456958bb415f220456e7659a985
I20260805 23:19:54.715839 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:54.723225 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227: Bootstrap starting.
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14365](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:54.725886 14449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:54.726886 14449 log.cc:824] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:54.729431 14449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227: No bootstrap required, opened a new log
I20260805 23:19:54.730834 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:54.716854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60337 (local address 127.23.88.126:44877)
0805 23:19:54.717264 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:54.717288 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:54.717324 (+ 36us) server_negotiation.cc:407] Connection header received
0805 23:19:54.717969 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:54.717984 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:54.718303 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:54.718577 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:54.719579 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:54.720560 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:54.721826 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:54.722320 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:54.722450 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:54.725395 (+ 2945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:54.725401 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:54.725413 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:54.725443 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:54.727535 (+ 2092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:54.728097 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:54.728104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:54.728109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:54.728244 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:54.728649 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:54.728652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:54.728654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:54.729193 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:54.729394 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:54.729790 (+ 396us) server_negotiation.cc:299] Negotiation successful
0805 23:19:54.730045 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":146,"threads_started":1}
I20260805 23:19:54.733737 14449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } }
I20260805 23:19:54.734026 14449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:54.734128 14449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017ee1cac994b259cc8d643fa8f2227, State: Initialized, Role: FOLLOWER
I20260805 23:19:54.734336 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:54.734591 14449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } }
I20260805 23:19:54.734750 14449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:54.734841 14449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:54.734943 14449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:54.736094 14449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } }
I20260805 23:19:54.736449 14449 leader_election.cc:302] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9017ee1cac994b259cc8d643fa8f2227; no voters:
I20260805 23:19:54.736797 14449 leader_election.cc:288] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:54.736896 14459 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:54.737114 14459 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [term 1 LEADER]: Becoming Leader. State: Replica: 9017ee1cac994b259cc8d643fa8f2227, State: Running, Role: LEADER
I20260805 23:19:54.737463 14459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } }
I20260805 23:19:54.738183 14449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:54.738639 14459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9017ee1cac994b259cc8d643fa8f2227. Latest consensus state: current_term: 1 leader_uuid: "9017ee1cac994b259cc8d643fa8f2227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } } }
I20260805 23:19:54.739190 14459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:54.740721 14461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9017ee1cac994b259cc8d643fa8f2227" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9017ee1cac994b259cc8d643fa8f2227" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 44877 } } }
I20260805 23:19:54.740862 14461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:54.741240 14468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:54.744760 14468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:54.746855 14453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:54.736665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51007 (local address 127.23.88.126:44877)
0805 23:19:54.736824 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:54.736831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:54.737165 (+ 334us) server_negotiation.cc:407] Connection header received
0805 23:19:54.737380 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:54.737385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:54.737431 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:54.737486 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:54.738391 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:54.739347 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:54.740638 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:54.741034 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:54.741138 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:54.742690 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:54.742695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:54.742697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:54.742725 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:54.744533 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:54.745209 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:54.745213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:54.745215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:54.745311 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:54.745743 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:54.745747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:54.745748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:54.745993 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:54.746149 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:54.746599 (+ 450us) server_negotiation.cc:299] Negotiation successful
0805 23:19:54.746708 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:19:54.751657 14468 catalog_manager.cc:1347] Generated new cluster ID: a7fcdd0f1b114fec8d1dde76587f183d
I20260805 23:19:54.751781 14468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:54.767947 14468 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:54.768913 14468 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:54.781404 14468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9017ee1cac994b259cc8d643fa8f2227: Generated new TSK 0
I20260805 23:19:54.782150 14468 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:54.789991 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14378
W20260805 23:19:54.797144 14363 connection.cc:535] client connection to 127.23.88.126:44877 recv error: Network error: failed to read from TLS socket (remote: 127.23.88.126:44877): Connection reset by peer (error 104)
2026-08-05T23:19:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14482](info): set up 2 sockets
Aug 05 23:19:54 dist-test-slave-vkjp krb5kdc[14482](info): commencing operation
krb5kdc: starting...
W20260805 23:19:55.865222 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:55 dist-test-slave-vkjp krb5kdc[14482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:55.994300 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41071
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:43915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:56.108819 14495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:56.109117 14495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:56.109167 14495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:56.113566 14495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:56.113639 14495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:56.113659 14495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:56.113675 14495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:56.113691 14495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:56.119247 14495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:43915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41071
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:56.120357 14495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:56.121420 14495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:56.129998 14502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:56.130100 14495 server_base.cc:1034] running on GCE node
W20260805 23:19:56.130002 14504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:56.129998 14501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:56.130892 14495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:56.132153 14495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:56.133347 14495 hybrid_clock.cc:648] HybridClock initialized: now 1785971996133316 us; error 45 us; skew 500 ppm
Aug 05 23:19:56 dist-test-slave-vkjp krb5kdc[14482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971996, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:56.136363 14495 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:56.137302 14495 webserver.cc:460] Webserver started at http://127.23.88.126:37895/ using document root <none> and password file <none>
I20260805 23:19:56.137959 14495 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:56.138057 14495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:56.138307 14495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:56.140442 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "5424a23b8df74411add70feeaac32b76"
format_stamp: "Formatted at 2026-08-05 23:19:56 on dist-test-slave-vkjp"
server_key: "94a5b81d5120dcf3c5845a6014a9ce26"
server_key_iv: "18433bf4bf07f366a1fa8024601f44a9"
server_key_version: "encryptionkey@0"
I20260805 23:19:56.141024 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "5424a23b8df74411add70feeaac32b76"
format_stamp: "Formatted at 2026-08-05 23:19:56 on dist-test-slave-vkjp"
server_key: "94a5b81d5120dcf3c5845a6014a9ce26"
server_key_iv: "18433bf4bf07f366a1fa8024601f44a9"
server_key_version: "encryptionkey@0"
I20260805 23:19:56.144956 14495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:19:56.147598 14512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:56.148897 14495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260805 23:19:56.149063 14495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "5424a23b8df74411add70feeaac32b76"
format_stamp: "Formatted at 2026-08-05 23:19:56 on dist-test-slave-vkjp"
server_key: "94a5b81d5120dcf3c5845a6014a9ce26"
server_key_iv: "18433bf4bf07f366a1fa8024601f44a9"
server_key_version: "encryptionkey@0"
I20260805 23:19:56.149189 14495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:56.164489 14495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:56.167951 14495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:56.168175 14495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:56.175702 14495 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41071
I20260805 23:19:56.175721 14566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41071 every 8 connection(s)
I20260805 23:19:56.176700 14495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:56.179505 14567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:56.181635 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14495
I20260805 23:19:56.181807 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:56.182093 23905 external_mini_cluster.cc:1428] Setting key be8f92377b0af6d9efae704a3e83e40c
I20260805 23:19:56.186116 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76: Bootstrap starting.
I20260805 23:19:56.188772 14567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:56.189674 14567 log.cc:824] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:56.192011 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:56.183614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41839 (local address 127.23.88.126:41071)
0805 23:19:56.184060 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:56.184081 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:56.184130 (+ 49us) server_negotiation.cc:407] Connection header received
0805 23:19:56.184989 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:56.185013 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:56.185434 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:56.185791 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:56.186209 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:56.187706 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:56.188622 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:56.189331 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:56.189483 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:56.190391 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:56.190396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:56.190408 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:56.190783 (+ 375us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:56.190839 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:56.190861 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:56.191001 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:56.191185 (+ 184us) server_negotiation.cc:299] Negotiation successful
0805 23:19:56.191333 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260805 23:19:56.192137 14567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76: No bootstrap required, opened a new log
I20260805 23:19:56.194876 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:56.196373 14567 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } }
I20260805 23:19:56.196636 14567 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:56.196735 14567 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5424a23b8df74411add70feeaac32b76, State: Initialized, Role: FOLLOWER
I20260805 23:19:56.197229 14567 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } }
I20260805 23:19:56.197431 14567 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:56.197531 14567 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:56.197645 14567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:56.198923 14567 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } }
I20260805 23:19:56.199334 14567 leader_election.cc:302] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5424a23b8df74411add70feeaac32b76; no voters:
I20260805 23:19:56.199704 14567 leader_election.cc:288] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:56.199808 14577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:56.200039 14577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [term 1 LEADER]: Becoming Leader. State: Replica: 5424a23b8df74411add70feeaac32b76, State: Running, Role: LEADER
I20260805 23:19:56.200428 14577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } }
I20260805 23:19:56.201186 14567 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:56.202044 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:56.197226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57219 (local address 127.23.88.126:41071)
0805 23:19:56.197374 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:56.197378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:56.197390 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:19:56.197423 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:56.197425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:56.197465 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:56.197537 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:56.197822 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:56.198677 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:56.199732 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:56.200148 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:56.200261 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:56.201438 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:56.201442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:56.201445 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:56.201459 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:56.201488 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:56.201504 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:56.201604 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:56.201822 (+ 218us) server_negotiation.cc:299] Negotiation successful
0805 23:19:56.201870 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:19:56.202100 14579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5424a23b8df74411add70feeaac32b76. Latest consensus state: current_term: 1 leader_uuid: "5424a23b8df74411add70feeaac32b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } } }
I20260805 23:19:56.202211 14579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:56.202603 14585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:56.203223 14578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5424a23b8df74411add70feeaac32b76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5424a23b8df74411add70feeaac32b76" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41071 } } }
I20260805 23:19:56.203320 14578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:56.205699 14585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:56.210789 14585 catalog_manager.cc:1347] Generated new cluster ID: abe9d787a7d642f19318a1888074f356
I20260805 23:19:56.210855 14585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:56.237326 14585 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:56.238461 14585 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:56.251349 14585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5424a23b8df74411add70feeaac32b76: Generated new TSK 0
I20260805 23:19:56.252208 14585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:56.260895 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14495
W20260805 23:19:56.267195 14480 connection.cc:535] client connection to 127.23.88.126:41071 recv error: Network error: failed to read from TLS socket (remote: 127.23.88.126:41071): Connection reset by peer (error 104)
2026-08-05T23:19:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:56 dist-test-slave-vkjp krb5kdc[14598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:56 dist-test-slave-vkjp krb5kdc[14598](info): set up 2 sockets
Aug 05 23:19:56 dist-test-slave-vkjp krb5kdc[14598](info): commencing operation
krb5kdc: starting...
W20260805 23:19:57.369230 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:57 dist-test-slave-vkjp krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:57.494424 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41689
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:57.611893 14614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:57.612226 14614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:57.612337 14614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:57.616545 14614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:57.616654 14614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:57.616694 14614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:57.616748 14614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:57.616802 14614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:57.622412 14614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:57.623615 14614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:57.624703 14614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:57.633540 14623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:57.633553 14620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:57.633534 14621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:57.633935 14614 server_base.cc:1034] running on GCE node
I20260805 23:19:57.634488 14614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:57.635723 14614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:57.636895 14614 hybrid_clock.cc:648] HybridClock initialized: now 1785971997636866 us; error 47 us; skew 500 ppm
Aug 05 23:19:57 dist-test-slave-vkjp krb5kdc[14598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971997, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:57.639765 14614 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:57.641088 14614 webserver.cc:460] Webserver started at http://127.23.88.126:41349/ using document root <none> and password file <none>
I20260805 23:19:57.641847 14614 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:57.641921 14614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:57.642185 14614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:57.644549 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "88d7b20308d444509e62052f720ccfdb"
format_stamp: "Formatted at 2026-08-05 23:19:57 on dist-test-slave-vkjp"
server_key: "9db9ba7aa90cb075871e41b68cb14578"
server_key_iv: "4f20de41ad6b629969c94ac0ea16e015"
server_key_version: "encryptionkey@0"
I20260805 23:19:57.645195 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "88d7b20308d444509e62052f720ccfdb"
format_stamp: "Formatted at 2026-08-05 23:19:57 on dist-test-slave-vkjp"
server_key: "9db9ba7aa90cb075871e41b68cb14578"
server_key_iv: "4f20de41ad6b629969c94ac0ea16e015"
server_key_version: "encryptionkey@0"
I20260805 23:19:57.649492 14614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260805 23:19:57.652302 14630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:57.653620 14614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:57.653852 14614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "88d7b20308d444509e62052f720ccfdb"
format_stamp: "Formatted at 2026-08-05 23:19:57 on dist-test-slave-vkjp"
server_key: "9db9ba7aa90cb075871e41b68cb14578"
server_key_iv: "4f20de41ad6b629969c94ac0ea16e015"
server_key_version: "encryptionkey@0"
I20260805 23:19:57.653987 14614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:57.658733 14614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:57.662421 14614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:57.662691 14614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:57.670881 14684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41689 every 8 connection(s)
I20260805 23:19:57.670892 14614 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41689
I20260805 23:19:57.672060 14614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:57.675026 14685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:57.680524 14685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb: Bootstrap starting.
I20260805 23:19:57.681354 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14614
I20260805 23:19:57.681533 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:57.681846 23905 external_mini_cluster.cc:1428] Setting key b793905083269a5fad346b9ca69b6f52
I20260805 23:19:57.683669 14685 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:57.684582 14685 log.cc:824] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:57.686844 14685 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb: No bootstrap required, opened a new log
I20260805 23:19:57.687386 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:57.683002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49357 (local address 127.23.88.126:41689)
0805 23:19:57.683521 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:57.683543 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:57.683587 (+ 44us) server_negotiation.cc:407] Connection header received
0805 23:19:57.684515 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:57.684539 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:57.684911 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:57.685299 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:57.686091 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:57.686095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:57.686106 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:57.686116 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:57.686166 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:57.686185 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:57.686307 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:57.686417 (+ 110us) server_negotiation.cc:299] Negotiation successful
0805 23:19:57.686581 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":140,"threads_started":1}
I20260805 23:19:57.690279 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:57.691165 14685 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } }
I20260805 23:19:57.691423 14685 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:57.691514 14685 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d7b20308d444509e62052f720ccfdb, State: Initialized, Role: FOLLOWER
I20260805 23:19:57.692008 14685 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } }
I20260805 23:19:57.692173 14685 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:57.692262 14685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:57.692365 14685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:57.693809 14685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } }
I20260805 23:19:57.694218 14685 leader_election.cc:302] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d7b20308d444509e62052f720ccfdb; no voters:
I20260805 23:19:57.694386 14689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:57.692589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38071 (local address 127.23.88.126:41689)
0805 23:19:57.692733 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:57.692738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:57.692932 (+ 194us) server_negotiation.cc:407] Connection header received
0805 23:19:57.693055 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:57.693057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:57.693090 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:57.693162 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:57.693932 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:57.693938 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:57.693941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:19:57.693955 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:57.693988 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:19:57.694004 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:57.694079 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:57.694171 (+ 92us) server_negotiation.cc:299] Negotiation successful
0805 23:19:57.694205 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:19:57.694626 14685 leader_election.cc:288] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:57.694697 14696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:57.694922 14696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [term 1 LEADER]: Becoming Leader. State: Replica: 88d7b20308d444509e62052f720ccfdb, State: Running, Role: LEADER
I20260805 23:19:57.695246 14696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } }
I20260805 23:19:57.695909 14685 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:57.696393 14696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d7b20308d444509e62052f720ccfdb. Latest consensus state: current_term: 1 leader_uuid: "88d7b20308d444509e62052f720ccfdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } } }
I20260805 23:19:57.696537 14696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:57.696823 14698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d7b20308d444509e62052f720ccfdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d7b20308d444509e62052f720ccfdb" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41689 } } }
I20260805 23:19:57.696928 14698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:57.696970 14702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:57.700148 14702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:57.706039 14702 catalog_manager.cc:1347] Generated new cluster ID: 556f758e6e1d46c1a73cff0d4eb7ecbd
I20260805 23:19:57.706113 14702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:57.714847 14702 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:57.715235 14702 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:57.720600 14702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d7b20308d444509e62052f720ccfdb: Generated new TSK 0
I20260805 23:19:57.721424 14702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:57.725827 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14614
2026-08-05T23:19:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:57 dist-test-slave-vkjp krb5kdc[14718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:57 dist-test-slave-vkjp krb5kdc[14718](info): set up 2 sockets
Aug 05 23:19:57 dist-test-slave-vkjp krb5kdc[14718](info): commencing operation
krb5kdc: starting...
W20260805 23:19:58.805856 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:19:58 dist-test-slave-vkjp krb5kdc[14718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:19:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:19:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:19:58.938457 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36881
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:36881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:19:59.056900 14731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:19:59.057246 14731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:19:59.057355 14731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:19:59.061586 14731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:19:59.061709 14731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:19:59.061749 14731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:19:59.061787 14731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:19:59.061848 14731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:19:59.067744 14731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:36881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:36881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:19:59.068984 14731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:19:59.070066 14731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:19:59.078965 14736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:59.079175 14731 server_base.cc:1034] running on GCE node
W20260805 23:19:59.078984 14737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:19:59.079005 14739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:19:59.079995 14731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:19:59.081218 14731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:19:59.082425 14731 hybrid_clock.cc:648] HybridClock initialized: now 1785971999082396 us; error 46 us; skew 500 ppm
Aug 05 23:19:59 dist-test-slave-vkjp krb5kdc[14718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971999, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:19:59.085824 14731 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:19:59.086856 14731 webserver.cc:460] Webserver started at http://127.23.88.126:45913/ using document root <none> and password file <none>
I20260805 23:19:59.087518 14731 fs_manager.cc:359] Metadata directory not provided
I20260805 23:19:59.087596 14731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:19:59.087800 14731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:19:59.090059 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "0a81bceee1e14497ab01a3ea2136d678"
format_stamp: "Formatted at 2026-08-05 23:19:59 on dist-test-slave-vkjp"
server_key: "c6d9f8cd58e9407ecdca1e7e1f882505"
server_key_iv: "d8b35ead0a64b18f851175e47054a9fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:59.090979 14731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "0a81bceee1e14497ab01a3ea2136d678"
format_stamp: "Formatted at 2026-08-05 23:19:59 on dist-test-slave-vkjp"
server_key: "c6d9f8cd58e9407ecdca1e7e1f882505"
server_key_iv: "d8b35ead0a64b18f851175e47054a9fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:59.095206 14731 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260805 23:19:59.097833 14746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:19:59.099311 14731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:19:59.099551 14731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "0a81bceee1e14497ab01a3ea2136d678"
format_stamp: "Formatted at 2026-08-05 23:19:59 on dist-test-slave-vkjp"
server_key: "c6d9f8cd58e9407ecdca1e7e1f882505"
server_key_iv: "d8b35ead0a64b18f851175e47054a9fd"
server_key_version: "encryptionkey@0"
I20260805 23:19:59.099687 14731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:19:59.124965 14731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:19:59.128830 14731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:19:59.129081 14731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:19:59.137105 14731 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:36881
I20260805 23:19:59.137118 14801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:36881 every 8 connection(s)
I20260805 23:19:59.138166 14731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:19:59.141306 14802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:19:59.146014 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14731
I20260805 23:19:59.146214 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:19:59.146581 23905 external_mini_cluster.cc:1428] Setting key ecf3d2e772c36a54e7e0345435a20f2f
I20260805 23:19:59.148120 14802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678: Bootstrap starting.
I20260805 23:19:59.150784 14802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678: Neither blocks nor log segments found. Creating new log.
I20260805 23:19:59.151713 14802 log.cc:824] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678: Log is configured to *not* fsync() on all Append() calls
I20260805 23:19:59.154300 14802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678: No bootstrap required, opened a new log
Aug 05 23:19:59 dist-test-slave-vkjp krb5kdc[14718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785971998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:19:59.158357 14802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } }
I20260805 23:19:59.158668 14802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:19:59.158753 14802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a81bceee1e14497ab01a3ea2136d678, State: Initialized, Role: FOLLOWER
I20260805 23:19:59.159328 14802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } }
I20260805 23:19:59.159511 14802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:19:59.159595 14802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:19:59.159726 14802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:19:59.160773 14802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } }
I20260805 23:19:59.161186 14802 leader_election.cc:302] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a81bceee1e14497ab01a3ea2136d678; no voters:
I20260805 23:19:59.161613 14802 leader_election.cc:288] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:19:59.161755 14807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:19:59.161976 14807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [term 1 LEADER]: Becoming Leader. State: Replica: 0a81bceee1e14497ab01a3ea2136d678, State: Running, Role: LEADER
I20260805 23:19:59.162315 14807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } }
I20260805 23:19:59.163004 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:59.148169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53203 (local address 127.23.88.126:36881)
0805 23:19:59.148699 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:59.148716 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:59.148756 (+ 40us) server_negotiation.cc:407] Connection header received
0805 23:19:59.149448 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:59.149466 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:59.149828 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:59.150122 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:59.151247 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:59.152541 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:59.153809 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:59.154316 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:59.154512 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:59.157257 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:59.157262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:59.157279 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:59.157313 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:59.159413 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:59.159982 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:59.159991 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:59.160002 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:59.160131 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:59.160447 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:59.160450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:59.160452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:59.160902 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:59.161163 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:59.161480 (+ 317us) server_negotiation.cc:299] Negotiation successful
0805 23:19:59.162047 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":160,"threads_started":1}
I20260805 23:19:59.163071 14802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:19:59.163642 14807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a81bceee1e14497ab01a3ea2136d678. Latest consensus state: current_term: 1 leader_uuid: "0a81bceee1e14497ab01a3ea2136d678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } } }
I20260805 23:19:59.163771 14807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:59.164495 14809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a81bceee1e14497ab01a3ea2136d678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a81bceee1e14497ab01a3ea2136d678" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 36881 } } }
I20260805 23:19:59.164613 14809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678 [sys.catalog]: This master's current role is: LEADER
I20260805 23:19:59.165968 14811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:19:59.166706 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:19:59.169656 14811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:19:59.176645 14811 catalog_manager.cc:1347] Generated new cluster ID: 1e2f7f8fac424238b14890623c832fbb
I20260805 23:19:59.176810 14811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:19:59.178526 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:19:59.169179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58829 (local address 127.23.88.126:36881)
0805 23:19:59.169298 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:19:59.169303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:19:59.169635 (+ 332us) server_negotiation.cc:407] Connection header received
0805 23:19:59.169752 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:19:59.169756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:19:59.169809 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:19:59.169907 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:19:59.170815 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:59.171622 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:59.172505 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:19:59.172771 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:19:59.172892 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:19:59.174604 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:19:59.174608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:19:59.174611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:19:59.174639 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:19:59.176524 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:59.177089 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:59.177091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:59.177093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:59.177166 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:19:59.177486 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:19:59.177491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:19:59.177492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:19:59.177696 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:19:59.177839 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0805 23:19:59.178103 (+ 264us) server_negotiation.cc:299] Negotiation successful
0805 23:19:59.178228 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:19:59.187120 14811 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:19:59.188583 14811 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:19:59.194109 14811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a81bceee1e14497ab01a3ea2136d678: Generated new TSK 0
I20260805 23:19:59.194882 14811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:19:59.203840 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14731
2026-08-05T23:19:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:19:59 dist-test-slave-vkjp krb5kdc[14836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:19:59 dist-test-slave-vkjp krb5kdc[14836](info): set up 2 sockets
Aug 05 23:19:59 dist-test-slave-vkjp krb5kdc[14836](info): commencing operation
krb5kdc: starting...
W20260805 23:20:00.301862 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:20:00 dist-test-slave-vkjp krb5kdc[14836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:20:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:20:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:20:00.437014 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45727
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:41109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:45727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:20:00.554227 14849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:00.554554 14849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:00.554652 14849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:00.559544 14849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:00.559638 14849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:00.559659 14849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:00.559676 14849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:00.559692 14849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:00.565574 14849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:41109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:45727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:45727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:00.566787 14849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:00.567925 14849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:00.576603 14854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:00.576675 14857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:00.576743 14855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:00.577044 14849 server_base.cc:1034] running on GCE node
I20260805 23:20:00.577608 14849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:00.578754 14849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:00.579959 14849 hybrid_clock.cc:648] HybridClock initialized: now 1785972000579910 us; error 64 us; skew 500 ppm
Aug 05 23:20:00 dist-test-slave-vkjp krb5kdc[14836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972000, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:00.589578 14849 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:20:00.590649 14849 webserver.cc:460] Webserver started at http://127.23.88.126:43237/ using document root <none> and password file <none>
I20260805 23:20:00.591320 14849 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:00.591423 14849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:00.591665 14849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:00.593863 14849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "2ab5eced189b4977a483871b026ad61a"
format_stamp: "Formatted at 2026-08-05 23:20:00 on dist-test-slave-vkjp"
server_key: "535a08d67eca5545f8fd76af335b61bf"
server_key_iv: "72d50da4fd6c992469cad61c5ef39f6f"
server_key_version: "encryptionkey@0"
I20260805 23:20:00.594491 14849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "2ab5eced189b4977a483871b026ad61a"
format_stamp: "Formatted at 2026-08-05 23:20:00 on dist-test-slave-vkjp"
server_key: "535a08d67eca5545f8fd76af335b61bf"
server_key_iv: "72d50da4fd6c992469cad61c5ef39f6f"
server_key_version: "encryptionkey@0"
I20260805 23:20:00.598423 14849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:20:00.601008 14864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:00.602291 14849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260805 23:20:00.602445 14849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "2ab5eced189b4977a483871b026ad61a"
format_stamp: "Formatted at 2026-08-05 23:20:00 on dist-test-slave-vkjp"
server_key: "535a08d67eca5545f8fd76af335b61bf"
server_key_iv: "72d50da4fd6c992469cad61c5ef39f6f"
server_key_version: "encryptionkey@0"
I20260805 23:20:00.602568 14849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:00.624735 14849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:00.628309 14849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:00.628547 14849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:00.636086 14849 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:45727
I20260805 23:20:00.636114 14920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:45727 every 8 connection(s)
I20260805 23:20:00.637081 14849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:20:00.639989 14921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:00.644855 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14849
I20260805 23:20:00.645020 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:20:00.645303 23905 external_mini_cluster.cc:1428] Setting key 797022fc54e07f6fd2d75c8519714b95
I20260805 23:20:00.645833 14921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a: Bootstrap starting.
I20260805 23:20:00.648438 14921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:00.649384 14921 log.cc:824] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:00.651790 14921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a: No bootstrap required, opened a new log
I20260805 23:20:00.654838 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:00.646748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50715 (local address 127.23.88.126:45727)
0805 23:20:00.647301 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:00.647317 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:00.647352 (+ 35us) server_negotiation.cc:407] Connection header received
0805 23:20:00.648021 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:00.648037 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:00.648381 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:00.648744 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:00.649090 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:00.650293 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:00.651515 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:00.652037 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:00.652170 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:00.653064 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:00.653069 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:00.653082 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:20:00.653523 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:00.653580 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:20:00.653602 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:00.653681 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:00.653851 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 23:20:00.654093 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":157,"threads_started":1}
I20260805 23:20:00.656036 14921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } }
I20260805 23:20:00.656315 14921 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:20:00.656383 14921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab5eced189b4977a483871b026ad61a, State: Initialized, Role: FOLLOWER
I20260805 23:20:00.656859 14921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } }
I20260805 23:20:00.657016 14921 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:20:00.657081 14921 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:20:00.657218 14921 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:00.657909 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:20:00.658591 14921 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } }
I20260805 23:20:00.658979 14921 leader_election.cc:302] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ab5eced189b4977a483871b026ad61a; no voters:
I20260805 23:20:00.659463 14921 leader_election.cc:288] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:20:00.659571 14928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:20:00.659864 14928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [term 1 LEADER]: Becoming Leader. State: Replica: 2ab5eced189b4977a483871b026ad61a, State: Running, Role: LEADER
I20260805 23:20:00.660279 14928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } }
I20260805 23:20:00.661039 14921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:00.662205 14932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ab5eced189b4977a483871b026ad61a. Latest consensus state: current_term: 1 leader_uuid: "2ab5eced189b4977a483871b026ad61a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } } }
I20260805 23:20:00.662647 14932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:00.662520 14931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ab5eced189b4977a483871b026ad61a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab5eced189b4977a483871b026ad61a" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 45727 } } }
I20260805 23:20:00.662959 14931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:00.663734 14941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:20:00.666450 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:00.660459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48981 (local address 127.23.88.126:45727)
0805 23:20:00.660578 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:00.660582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:00.661853 (+ 1271us) server_negotiation.cc:407] Connection header received
0805 23:20:00.662076 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:00.662080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:00.662130 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:00.662189 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:00.662643 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:00.663430 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:00.664559 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:00.664891 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:00.665015 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:00.665947 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:00.665952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:00.665955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:20:00.665970 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:00.666004 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:20:00.666018 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:00.666101 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:00.666230 (+ 129us) server_negotiation.cc:299] Negotiation successful
0805 23:20:00.666295 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260805 23:20:00.667608 14941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:20:00.673843 14941 catalog_manager.cc:1347] Generated new cluster ID: a9cbba31a501426dbb0a464b950c9d95
I20260805 23:20:00.673951 14941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:20:00.685274 14941 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:20:00.686312 14941 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:20:00.690928 14941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ab5eced189b4977a483871b026ad61a: Generated new TSK 0
I20260805 23:20:00.691730 14941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:20:00.704363 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14849
2026-08-05T23:20:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:20:00 dist-test-slave-vkjp krb5kdc[14953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:20:00 dist-test-slave-vkjp krb5kdc[14953](info): set up 2 sockets
Aug 05 23:20:00 dist-test-slave-vkjp krb5kdc[14953](info): commencing operation
krb5kdc: starting...
W20260805 23:20:01.786231 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:20:01 dist-test-slave-vkjp krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:20:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:20:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:20:01.942941 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41213
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:46549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:41213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:20:02.078377 14967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:02.078718 14967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:02.078771 14967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:02.083281 14967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:02.083386 14967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:02.083411 14967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:02.083432 14967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:02.083482 14967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:02.089715 14967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:46549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:41213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:41213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:02.091038 14967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:02.092404 14967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:02.103386 14977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:02.103520 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:02.103386 14974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:02.103892 14967 server_base.cc:1034] running on GCE node
I20260805 23:20:02.104938 14967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:02.106824 14967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:02.108162 14967 hybrid_clock.cc:648] HybridClock initialized: now 1785972002108121 us; error 71 us; skew 500 ppm
Aug 05 23:20:02 dist-test-slave-vkjp krb5kdc[14953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972002, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:02.112502 14967 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:20:02.113919 14967 webserver.cc:460] Webserver started at http://127.23.88.126:39827/ using document root <none> and password file <none>
I20260805 23:20:02.114619 14967 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:02.114748 14967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:02.115065 14967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:02.117707 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "17e12119928f454996bfaa8066b8782f"
format_stamp: "Formatted at 2026-08-05 23:20:02 on dist-test-slave-vkjp"
server_key: "53abcc7c9c73267bd6e23375ebade6fa"
server_key_iv: "a1483af93a1628cb6efab1262096e0f8"
server_key_version: "encryptionkey@0"
I20260805 23:20:02.118499 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "17e12119928f454996bfaa8066b8782f"
format_stamp: "Formatted at 2026-08-05 23:20:02 on dist-test-slave-vkjp"
server_key: "53abcc7c9c73267bd6e23375ebade6fa"
server_key_iv: "a1483af93a1628cb6efab1262096e0f8"
server_key_version: "encryptionkey@0"
I20260805 23:20:02.123569 14967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260805 23:20:02.127283 14985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:02.129163 14967 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:20:02.129489 14967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "17e12119928f454996bfaa8066b8782f"
format_stamp: "Formatted at 2026-08-05 23:20:02 on dist-test-slave-vkjp"
server_key: "53abcc7c9c73267bd6e23375ebade6fa"
server_key_iv: "a1483af93a1628cb6efab1262096e0f8"
server_key_version: "encryptionkey@0"
I20260805 23:20:02.129673 14967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:02.135833 14967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:02.139843 14967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:02.140090 14967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:02.148504 14967 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:41213
I20260805 23:20:02.148535 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:41213 every 8 connection(s)
I20260805 23:20:02.149593 14967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:20:02.151810 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 14967
I20260805 23:20:02.151994 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:20:02.152379 23905 external_mini_cluster.cc:1428] Setting key 7981e656b6590c51fcc8195fc187ccd0
I20260805 23:20:02.153189 15039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:02.158445 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:02.154043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59369 (local address 127.23.88.126:41213)
0805 23:20:02.154572 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:02.154589 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:02.154636 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:20:02.155488 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:02.155507 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:02.155819 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:02.156186 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:02.157158 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:02.157163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:02.157175 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:20:02.157186 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:02.157247 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:20:02.157268 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:02.157437 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:02.157510 (+ 73us) server_negotiation.cc:299] Negotiation successful
0805 23:20:02.157679 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":182,"threads_started":1}
I20260805 23:20:02.161029 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f: Bootstrap starting.
I20260805 23:20:02.161868 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260805 23:20:02.163913 15039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:02.164955 15039 log.cc:824] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:02.166491 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:02.164375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38529 (local address 127.23.88.126:41213)
0805 23:20:02.164532 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:02.164537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:02.164782 (+ 245us) server_negotiation.cc:407] Connection header received
0805 23:20:02.164937 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:02.164939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:02.164978 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:02.165087 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:02.165977 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:02.165983 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:02.165987 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0805 23:20:02.166003 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:02.166044 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0805 23:20:02.166058 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:02.166138 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:02.166314 (+ 176us) server_negotiation.cc:299] Negotiation successful
0805 23:20:02.166352 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:20:02.167955 15039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f: No bootstrap required, opened a new log
I20260805 23:20:02.172206 15039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } }
I20260805 23:20:02.172533 15039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260805 23:20:02.172626 15039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e12119928f454996bfaa8066b8782f, State: Initialized, Role: FOLLOWER
I20260805 23:20:02.173152 15039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } }
I20260805 23:20:02.173332 15039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260805 23:20:02.173436 15039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260805 23:20:02.173578 15039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:02.174896 15039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } }
I20260805 23:20:02.175340 15039 leader_election.cc:302] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17e12119928f454996bfaa8066b8782f; no voters:
I20260805 23:20:02.175720 15039 leader_election.cc:288] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260805 23:20:02.175904 15049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:20:02.176203 15049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [term 1 LEADER]: Becoming Leader. State: Replica: 17e12119928f454996bfaa8066b8782f, State: Running, Role: LEADER
I20260805 23:20:02.176597 15049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } }
I20260805 23:20:02.177191 15039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:02.178591 15051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17e12119928f454996bfaa8066b8782f. Latest consensus state: current_term: 1 leader_uuid: "17e12119928f454996bfaa8066b8782f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } } }
I20260805 23:20:02.178784 15051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:02.178587 15050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17e12119928f454996bfaa8066b8782f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e12119928f454996bfaa8066b8782f" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 41213 } } }
I20260805 23:20:02.178992 15050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:02.179349 15059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:20:02.183307 15059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:20:02.189565 15059 catalog_manager.cc:1347] Generated new cluster ID: 45a804e90e7d48b7b7dd1c3da395a739
I20260805 23:20:02.189669 15059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:20:02.208897 15059 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:20:02.209365 15059 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:20:02.226938 15059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17e12119928f454996bfaa8066b8782f: Generated new TSK 0
I20260805 23:20:02.228034 15059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260805 23:20:02.233394 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 14967
2026-08-05T23:20:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1529 ms)
[----------] 18 tests from AuthTokenIssuingTest (26559 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:20:02 dist-test-slave-vkjp krb5kdc[15070](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:20:02 dist-test-slave-vkjp krb5kdc[15070](info): set up 2 sockets
Aug 05 23:20:02 dist-test-slave-vkjp krb5kdc[15070](info): commencing operation
krb5kdc: starting...
W20260805 23:20:03.359588 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:20:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:20:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:20:03.498701 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37647
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:20:03.619680 15083 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:03.620064 15083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:03.620138 15083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:03.620242 15083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:03.624605 15083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:03.624722 15083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:03.624754 15083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:03.624785 15083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:03.624823 15083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:03.631017 15083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:37647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:03.632349 15083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:03.633546 15083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:03.642692 15091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:03.642707 15089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:03.642865 15083 server_base.cc:1034] running on GCE node
W20260805 23:20:03.642707 15088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:03.643748 15083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:03.645043 15083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:03.646275 15083 hybrid_clock.cc:648] HybridClock initialized: now 1785972003646257 us; error 44 us; skew 500 ppm
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:03.649528 15083 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:20:03.650586 15083 webserver.cc:460] Webserver started at http://127.23.88.126:41359/ using document root <none> and password file <none>
I20260805 23:20:03.651239 15083 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:03.651347 15083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:03.651674 15083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:03.653868 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "eb9dafb87bd84a3486018269f8791cbc"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "dcb39a97f49fedaee527ac1953faae65"
server_key_iv: "57bf39428c779bd0b27cf554eddcb8ff"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.654489 15083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "eb9dafb87bd84a3486018269f8791cbc"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "dcb39a97f49fedaee527ac1953faae65"
server_key_iv: "57bf39428c779bd0b27cf554eddcb8ff"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.658712 15083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260805 23:20:03.661553 15101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:03.662992 15083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260805 23:20:03.663192 15083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "eb9dafb87bd84a3486018269f8791cbc"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "dcb39a97f49fedaee527ac1953faae65"
server_key_iv: "57bf39428c779bd0b27cf554eddcb8ff"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.663339 15083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:03.684934 15083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:03.688959 15083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:03.689206 15083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:03.697926 15083 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:37647
I20260805 23:20:03.698239 15154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:37647 every 8 connection(s)
I20260805 23:20:03.699133 15083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:20:03.702090 15156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:03.706017 15156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:03.706621 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15083
I20260805 23:20:03.706761 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:20:03.707055 23905 external_mini_cluster.cc:1428] Setting key f699b0bddeb5c784cf0d863379d0844f
WARNING: no policy specified for kudu/127.23.88.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.125@KRBTEST.COM" created.
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:20:03.726104 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.708863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37647 (local address 127.0.0.1:39420)
0805 23:20:03.709490 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.709529 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:03.710413 (+ 884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:03.711108 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:03.711114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:03.711573 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:03.711812 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.711832 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.712987 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.712989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.713700 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.713709 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.714355 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.714359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.714468 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.715224 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:03.715241 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:03.720220 (+ 4979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:03.722677 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.722687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.722696 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.722956 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.723396 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.723399 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.723401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.723530 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.724430 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:03.724436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:03.724599 (+ 163us) client_negotiation.cc:769] Sending connection context
0805 23:20:03.724789 (+ 190us) client_negotiation.cc:240] Negotiation successful
0805 23:20:03.725111 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"spinlock_wait_cycles":8448,"thread_start_us":199,"threads_started":1}
I20260805 23:20:03.726106 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.708863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39420 (local address 127.23.88.126:37647)
0805 23:20:03.709342 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:03.709364 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:03.709645 (+ 281us) server_negotiation.cc:407] Connection header received
0805 23:20:03.710719 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:03.710723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:03.710898 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:03.711091 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:03.711985 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.712843 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.713821 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.714183 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.714361 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.720448 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:03.720459 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:03.720478 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:03.720515 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:03.722535 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.723109 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.723118 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.723122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.723281 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.723641 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.723647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.723652 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.724259 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:03.724411 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:03.724916 (+ 505us) server_negotiation.cc:299] Negotiation successful
0805 23:20:03.725258 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":304,"thread_start_us":147,"threads_started":1}
I20260805 23:20:03.729007 15156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:03.729761 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.729456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36276)
0805 23:20:03.729587 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.729696 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260805 23:20:03.729998 15103 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
I20260805 23:20:03.730904 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.730681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36282)
0805 23:20:03.730786 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.730852 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260805 23:20:03.731114 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34451: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
I20260805 23:20:03.738417 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:34451
--webserver_interface=127.23.88.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:20:03.780401 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } attempt: 1
I20260805 23:20:03.781376 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.781050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36292)
0805 23:20:03.781216 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.781290 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260805 23:20:03.782210 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.781996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36308)
0805 23:20:03.782103 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.782149 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:20:03.782428 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34451: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
I20260805 23:20:03.850616 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } attempt: 2
I20260805 23:20:03.851588 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.851253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36310)
0805 23:20:03.851443 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.851510 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260805 23:20:03.852559 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.852336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36318)
0805 23:20:03.852451 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.852499 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:20:03.852706 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:34451: Network error: Client connection negotiation failed: client connection to 127.23.88.125:34451: connect: Connection refused (error 111)
W20260805 23:20:03.855448 15163 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:03.855731 15163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:03.855809 15163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:03.855916 15163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:03.860173 15163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:03.860265 15163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:03.860286 15163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:03.860303 15163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:03.860338 15163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:03.866140 15163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:34451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.88.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:03.867523 15163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:03.868718 15163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:03.879367 15169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:03.879336 15171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:03.879336 15168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:03.879614 15163 server_base.cc:1034] running on GCE node
I20260805 23:20:03.880455 15163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:03.881827 15163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:03.883070 15163 hybrid_clock.cc:648] HybridClock initialized: now 1785972003883045 us; error 35 us; skew 500 ppm
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:03.886324 15163 init.cc:377] Logged in from keytab as kudu/127.23.88.125@KRBTEST.COM (short username kudu)
I20260805 23:20:03.887588 15163 webserver.cc:460] Webserver started at http://127.23.88.125:41719/ using document root <none> and password file <none>
I20260805 23:20:03.888226 15163 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:03.888334 15163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:03.888603 15163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:03.891039 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data/instance:
uuid: "97e957c4f0e241f58d2727b50403e3f8"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "414bcc57596c703e28084bf72abf25e1"
server_key_iv: "fb078a7c74037dd023ef22ec45fd27c5"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.891772 15163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal/instance:
uuid: "97e957c4f0e241f58d2727b50403e3f8"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "414bcc57596c703e28084bf72abf25e1"
server_key_iv: "fb078a7c74037dd023ef22ec45fd27c5"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.896222 15163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:20:03.899010 15178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:03.900421 15163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:20:03.900580 15163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "97e957c4f0e241f58d2727b50403e3f8"
format_stamp: "Formatted at 2026-08-05 23:20:03 on dist-test-slave-vkjp"
server_key: "414bcc57596c703e28084bf72abf25e1"
server_key_iv: "fb078a7c74037dd023ef22ec45fd27c5"
server_key_version: "encryptionkey@0"
I20260805 23:20:03.900745 15163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:03.911482 15163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:03.914742 15163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:03.914978 15163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:03.922675 15163 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.125:34451
I20260805 23:20:03.923039 15232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.125:34451 every 8 connection(s)
I20260805 23:20:03.923868 15163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
I20260805 23:20:03.925796 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15163
I20260805 23:20:03.925962 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-1/wal/instance
I20260805 23:20:03.926261 23905 external_mini_cluster.cc:1428] Setting key 6b61e67d73465a14022261dd00950fcb
I20260805 23:20:03.927280 15233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:03.931722 15233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.88.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.124@KRBTEST.COM" created.
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:20:03.943890 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } attempt: 3
I20260805 23:20:03.950002 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.934026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39426 (local address 127.23.88.126:37647)
0805 23:20:03.934208 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:03.934215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:03.935332 (+ 1117us) server_negotiation.cc:407] Connection header received
0805 23:20:03.936409 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:03.936412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:03.936468 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:03.936556 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:03.938012 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.938878 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.940766 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.941235 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.941388 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.944603 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:03.944607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:03.944610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:03.944640 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:03.946572 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.947169 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.947175 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.947181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.947260 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.947743 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.947746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.947747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.948013 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:03.948103 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:03.949676 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0805 23:20:03.949818 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260805 23:20:03.950407 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.934459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37647 (local address 127.0.0.1:39426)
0805 23:20:03.935056 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.935154 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:03.936119 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:03.936736 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:03.936744 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:03.937220 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:03.937826 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.937842 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.939072 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.939078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.940603 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.940614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.941454 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.941457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.941624 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.942330 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:03.942352 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:03.944376 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:03.946699 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.946707 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.946723 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.947017 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.947376 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.947379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.947381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.947594 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.948129 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:03.948137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:03.948498 (+ 361us) client_negotiation.cc:769] Sending connection context
0805 23:20:03.948717 (+ 219us) client_negotiation.cc:240] Negotiation successful
0805 23:20:03.949178 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":158,"threads_started":1}
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:03.952229 15233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:03.956861 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.944439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36334)
0805 23:20:03.944671 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.944687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:03.944777 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:03.945688 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:03.945694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:03.945877 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:03.945996 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.946002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.947519 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.947522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.948365 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.948376 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.949169 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.949173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.949290 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.949849 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:03.949861 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:03.951414 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:03.954667 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.954672 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.954674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.955002 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.955475 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.955480 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.955485 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.955598 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.956224 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:03.956227 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:03.956324 (+ 97us) client_negotiation.cc:769] Sending connection context
0805 23:20:03.956500 (+ 176us) client_negotiation.cc:240] Negotiation successful
0805 23:20:03.956668 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260805 23:20:03.956934 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.944520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.23.88.125:34451)
0805 23:20:03.945083 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:03.945103 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:03.945130 (+ 27us) server_negotiation.cc:407] Connection header received
0805 23:20:03.945182 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:03.945190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:03.945526 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:03.945701 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:03.946126 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.947404 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.948504 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.949007 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.949167 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.951695 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:03.951704 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:03.951717 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:03.951746 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:03.954528 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.955160 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.955171 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.955173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.955287 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.955739 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.955743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.955744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.956103 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:03.956242 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:03.956613 (+ 371us) server_negotiation.cc:299] Negotiation successful
0805 23:20:03.956747 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":173,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
I20260805 23:20:03.958864 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:39037
--webserver_interface=127.23.88.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:44013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 05 23:20:03 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:03.959589 15156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:03.960793 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.960126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51110)
0805 23:20:03.960629 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.960699 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423}
I20260805 23:20:03.961612 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.961332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51120)
0805 23:20:03.961474 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.961530 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260805 23:20:03.961769 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:03.964278 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.953120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.23.88.125:34451)
0805 23:20:03.953726 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:03.953733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:03.953746 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:20:03.953820 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:03.953826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:03.953884 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:03.953963 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:03.954558 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.955454 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.956511 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.956852 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.957268 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.960136 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:03.960142 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:03.960149 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:03.960181 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:03.962263 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.962782 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.962784 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.962785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.962860 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:03.963207 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:03.963209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:03.963210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:03.963471 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:03.963554 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:03.963916 (+ 362us) server_negotiation.cc:299] Negotiation successful
0805 23:20:03.964136 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":98,"server-negotiator.queue_time_us":529,"thread_start_us":124,"threads_started":1}
I20260805 23:20:03.964313 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.953045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36342)
0805 23:20:03.953203 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.953221 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:03.953324 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:03.954015 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:03.954018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:03.954288 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:03.954420 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.954428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.955561 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.955563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.956349 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:03.956357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:03.957246 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:03.957253 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:03.957360 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:03.957979 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:03.957994 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:03.959519 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:03.962390 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.962397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.962402 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.962679 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.962956 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:03.962962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:03.962967 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:03.963077 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:03.963581 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:03.963586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:03.963718 (+ 132us) client_negotiation.cc:769] Sending connection context
0805 23:20:03.963857 (+ 139us) client_negotiation.cc:240] Negotiation successful
0805 23:20:03.964052 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260805 23:20:03.965197 15233 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:03.966101 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.965729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51134)
0805 23:20:03.965883 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.966002 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260805 23:20:03.966375 15182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:03.967518 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.967214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51142)
0805 23:20:03.967355 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.967429 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260805 23:20:03.967736 15233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:03.992960 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 1
I20260805 23:20:03.994192 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.993719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51156)
0805 23:20:03.994020 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.994100 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260805 23:20:03.995272 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:03.994965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51164)
0805 23:20:03.995140 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:03.995201 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260805 23:20:03.995426 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:04.017055 15233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 1
I20260805 23:20:04.018054 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.017677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51172)
0805 23:20:04.017881 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.017956 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260805 23:20:04.018873 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.018626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51174)
0805 23:20:04.018740 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.018803 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:20:04.019064 15233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:04.070611 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 2
I20260805 23:20:04.071533 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.071221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51186)
0805 23:20:04.071375 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.071452 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260805 23:20:04.072381 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.072135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51194)
0805 23:20:04.072254 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.072306 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:20:04.072567 15156 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
W20260805 23:20:04.080569 15240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:04.080892 15240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:04.080945 15240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:04.081041 15240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:04.085534 15240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:04.085618 15240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:04.085641 15240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:04.085659 15240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:04.085719 15240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:04.087323 15233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 2
I20260805 23:20:04.088208 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.087863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51198)
0805 23:20:04.088036 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.088110 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260805 23:20:04.089043 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.088797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51200)
0805 23:20:04.088923 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.088977 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260805 23:20:04.089252 15233 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:39037: Network error: Client connection negotiation failed: client connection to 127.23.88.124:39037: connect: Connection refused (error 111)
I20260805 23:20:04.092252 15240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:44013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:37647,127.23.88.125:34451,127.23.88.124:39037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:39037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.88.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:04.093410 15240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:04.094467 15240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:04.104166 15249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:04.104182 15247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:04.104275 15240 server_base.cc:1034] running on GCE node
W20260805 23:20:04.104168 15246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:04.105134 15240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:04.106307 15240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:04.107491 15240 hybrid_clock.cc:648] HybridClock initialized: now 1785972004107453 us; error 52 us; skew 500 ppm
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:04.110335 15240 init.cc:377] Logged in from keytab as kudu/127.23.88.124@KRBTEST.COM (short username kudu)
I20260805 23:20:04.111462 15240 webserver.cc:460] Webserver started at http://127.23.88.124:46567/ using document root <none> and password file <none>
I20260805 23:20:04.112064 15240 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:04.112133 15240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:04.112370 15240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:04.114591 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data/instance:
uuid: "9de512fef63b4dde8b08ee597535da0e"
format_stamp: "Formatted at 2026-08-05 23:20:04 on dist-test-slave-vkjp"
server_key: "4f8222b8cafac7df2e21f62ab398191e"
server_key_iv: "f50f4a8612723e01762f3106f4247a7d"
server_key_version: "encryptionkey@0"
I20260805 23:20:04.115316 15240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal/instance:
uuid: "9de512fef63b4dde8b08ee597535da0e"
format_stamp: "Formatted at 2026-08-05 23:20:04 on dist-test-slave-vkjp"
server_key: "4f8222b8cafac7df2e21f62ab398191e"
server_key_iv: "f50f4a8612723e01762f3106f4247a7d"
server_key_version: "encryptionkey@0"
I20260805 23:20:04.119617 15240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:20:04.122262 15256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:04.123801 15240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:20:04.123968 15240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "9de512fef63b4dde8b08ee597535da0e"
format_stamp: "Formatted at 2026-08-05 23:20:04 on dist-test-slave-vkjp"
server_key: "4f8222b8cafac7df2e21f62ab398191e"
server_key_iv: "f50f4a8612723e01762f3106f4247a7d"
server_key_version: "encryptionkey@0"
I20260805 23:20:04.124099 15240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:04.138335 15240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:04.141860 15240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:04.142113 15240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:04.150046 15240 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.124:39037
I20260805 23:20:04.150390 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.124:39037 every 8 connection(s)
I20260805 23:20:04.151163 15240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
I20260805 23:20:04.154369 15312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:04.157230 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15240
I20260805 23:20:04.157428 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785971715702001-23905-0/minicluster-data/master-2/wal/instance
I20260805 23:20:04.157714 23905 external_mini_cluster.cc:1428] Setting key 65a80892e0d0edf5040bdc0099b23334
I20260805 23:20:04.159171 15312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:04.171728 15156 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 3
I20260805 23:20:04.172731 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.161240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39442 (local address 127.23.88.126:37647)
0805 23:20:04.161369 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.161373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.161730 (+ 357us) server_negotiation.cc:407] Connection header received
0805 23:20:04.161949 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.161952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.162006 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.162095 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.163049 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.163711 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.164755 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.164980 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.165071 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.168213 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.168220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.168224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.168259 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.170200 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.170849 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.170852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.170853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.170942 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.171372 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.171378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.171382 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.171684 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.171789 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.172389 (+ 600us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.172534 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260805 23:20:04.174443 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.161176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:37647 (local address 127.0.0.1:39442)
0805 23:20:04.161608 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.161658 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:04.161818 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:04.162150 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:04.162155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:04.162667 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:04.162879 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.162894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.163856 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.163858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.164621 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.164629 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.165143 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.165148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.165288 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.166166 (+ 878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:04.166182 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:04.168011 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:04.170365 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.170373 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.170383 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.170724 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.171045 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.171050 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.171054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.171262 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.171844 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:04.171850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:04.172113 (+ 263us) client_negotiation.cc:769] Sending connection context
0805 23:20:04.172945 (+ 832us) client_negotiation.cc:240] Negotiation successful
0805 23:20:04.173323 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":26,"spinlock_wait_cycles":6528,"thread_start_us":118,"threads_started":1}
I20260805 23:20:04.174443 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.159392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.23.88.124:39037)
0805 23:20:04.159903 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.159919 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.159957 (+ 38us) server_negotiation.cc:407] Connection header received
0805 23:20:04.160624 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.160641 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.160974 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.161367 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.162329 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.163628 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.164627 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.165085 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.165286 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.168561 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.168572 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.168593 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.168624 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.170601 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.171177 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.171190 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.171191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.171268 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.171580 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.171584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.171585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.172113 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.173067 (+ 954us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.173173 (+ 106us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.173390 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":357,"spinlock_wait_cycles":3200,"thread_start_us":155,"threads_started":1}
I20260805 23:20:04.176695 15312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:04.180496 15233 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } attempt: 3
I20260805 23:20:04.182992 15160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.172239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51218)
0805 23:20:04.172457 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.172474 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:04.172559 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:04.173026 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:04.173029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:04.173187 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:04.173275 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.173280 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.174074 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.174080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.174716 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.174730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.175322 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.175326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.175412 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.175966 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:04.175980 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:04.177705 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:04.181310 (+ 3605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.181313 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.181315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.181602 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.181911 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.181914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.181916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.182016 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.182488 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:04.182491 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:04.182589 (+ 98us) client_negotiation.cc:769] Sending connection context
0805 23:20:04.182672 (+ 83us) client_negotiation.cc:240] Negotiation successful
0805 23:20:04.182820 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:20:04.183706 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.172312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.23.88.124:39037)
0805 23:20:04.172751 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.172757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.172768 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:20:04.172831 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.172834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.172892 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.173367 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.173444 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.173961 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.174857 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.175207 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.175299 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.177852 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.177856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.177859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.177885 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.181168 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.181706 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.181708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.181710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.181781 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.182140 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.182144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.182146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.182375 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.183326 (+ 951us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.183403 (+ 77us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.183525 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":137,"threads_started":1}
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:04.188503 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.177575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.23.88.126:37647)
0805 23:20:04.177691 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.177696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.177707 (+ 11us) server_negotiation.cc:407] Connection header received
0805 23:20:04.177746 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.177748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.177787 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.177873 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.178636 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.179216 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.180197 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.180435 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.180551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.184217 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.184221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.184223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.184249 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.186136 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.186627 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.186630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.186632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.186730 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.187057 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.187060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.187062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.187340 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.187528 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.188223 (+ 695us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.188346 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:20:04.188735 15156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: Bootstrap starting.
I20260805 23:20:04.190377 15239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.177455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36346 (local address 127.23.88.125:34451)
0805 23:20:04.177942 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.177948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.177964 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:20:04.178007 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.178009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.178058 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.178156 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.178630 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.179557 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.180350 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.180768 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.180904 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.184604 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.184609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.184615 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.184649 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.187920 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.188868 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.188872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.188874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.188970 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189299 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.189302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.189304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.189602 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.189673 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.190081 (+ 408us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.190217 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382}
I20260805 23:20:04.190570 15236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.181229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51226)
0805 23:20:04.181375 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.181390 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:04.181465 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:04.181880 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:04.181887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:04.182107 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:04.182222 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.182229 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.183243 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.183247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.184028 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.184038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.184697 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.184701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.184794 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.185320 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:04.185331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:04.186829 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:04.189006 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.189015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.189264 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.189577 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.189582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.189666 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.190063 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:04.190066 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:04.190161 (+ 95us) client_negotiation.cc:769] Sending connection context
0805 23:20:04.190246 (+ 85us) client_negotiation.cc:240] Negotiation successful
0805 23:20:04.190371 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":3}
I20260805 23:20:04.191470 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.181499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.23.88.124:39037)
0805 23:20:04.181634 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.181640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.181654 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:20:04.181688 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.181691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.181733 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.181829 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.182362 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.183106 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.184195 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.184577 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.184692 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.186973 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.186980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.186986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.187019 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.188883 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189396 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.189399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.189401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.189472 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189769 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.189772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.189773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.189958 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.190643 (+ 685us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.190958 (+ 315us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.191271 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260805 23:20:04.191501 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.177238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:34451 (local address 127.0.0.1:36346)
0805 23:20:04.177413 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.177430 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:04.177509 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:04.178192 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:04.178198 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:04.178386 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:04.178517 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.178523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.179696 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.179699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.180217 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.180224 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.180906 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.180910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.181056 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.181959 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:04.181977 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:04.184451 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:04.188059 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.188065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.188070 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.188441 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.189093 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.189097 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.189099 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.189187 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.189752 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:04.189758 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:04.189869 (+ 111us) client_negotiation.cc:769] Sending connection context
0805 23:20:04.191017 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0805 23:20:04.191361 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":80}
I20260805 23:20:04.192133 15156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:04.193058 15156 log.cc:824] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:04.194419 15233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: Bootstrap starting.
I20260805 23:20:04.195554 15156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: No bootstrap required, opened a new log
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:04.196808 15233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:04.197535 15233 log.cc:824] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:04.198849 15312 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:04.199900 15233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: No bootstrap required, opened a new log
I20260805 23:20:04.200590 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.189803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36354 (local address 127.23.88.125:34451)
0805 23:20:04.189917 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.189921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.190183 (+ 262us) server_negotiation.cc:407] Connection header received
0805 23:20:04.190368 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.190371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.190415 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.190462 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.191376 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.192092 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.193199 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.193530 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.193607 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.196690 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.196694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.196696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.196724 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.198476 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.199018 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.199021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.199023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.199145 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.199488 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.199492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.199493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.199747 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.199926 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.200296 (+ 370us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.200429 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260805 23:20:04.201256 15156 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.201519 15156 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9dafb87bd84a3486018269f8791cbc, State: Initialized, Role: FOLLOWER
I20260805 23:20:04.201871 15156 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.203446 15156 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:04.204830 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 05 23:20:04 dist-test-slave-vkjp krb5kdc[15070](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972004, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:04.205363 15233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.205607 15233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e957c4f0e241f58d2727b50403e3f8, State: Initialized, Role: FOLLOWER
I20260805 23:20:04.206058 15233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.206586 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.206787 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.207154 15129 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eb9dafb87bd84a3486018269f8791cbc"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39452
I20260805 23:20:04.207479 15129 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 23:20:04.207593 15233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:04.207593 15129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:04.209133 15129 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
W20260805 23:20:04.209772 15333 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:20:04.209842 15333 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:04.209812 15321 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.209947 15321 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.211902 15129 leader_election.cc:288] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [CANDIDATE]: Term 1 election: Requested vote from peers 97e957c4f0e241f58d2727b50403e3f8 (127.23.88.125:34451), 9de512fef63b4dde8b08ee597535da0e (127.23.88.124:39037)
I20260805 23:20:04.212904 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.199437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:39037 (local address 127.0.0.1:51228)
0805 23:20:04.199583 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:04.199599 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:04.199688 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:04.200066 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:04.200069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:04.200257 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:04.200353 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.200361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.201654 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.201658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.202476 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:04.202487 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.203387 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.203391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:04.203478 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.204015 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:04.204034 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:04.205822 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:04.210708 (+ 4886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.210712 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.210714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.211003 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.211397 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:04.211400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:04.211402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:04.211506 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:04.212368 (+ 862us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:04.212371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:04.212468 (+ 97us) client_negotiation.cc:769] Sending connection context
0805 23:20:04.212573 (+ 105us) client_negotiation.cc:240] Negotiation successful
0805 23:20:04.212741 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260805 23:20:04.211580 15206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb9dafb87bd84a3486018269f8791cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "97e957c4f0e241f58d2727b50403e3f8"
I20260805 23:20:04.213145 15206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:04.213771 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.199589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.23.88.124:39037)
0805 23:20:04.199689 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.199693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.199707 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:20:04.199784 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.199787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.199839 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.199902 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.200469 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.201226 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.202717 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.203163 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.203264 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.206053 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.206057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.206059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.206086 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.209475 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.211173 (+ 1698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.211177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.211179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.211285 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.211621 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.211627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.211632 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.212030 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.212128 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.213507 (+ 1379us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.213632 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260805 23:20:04.214676 15206 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb9dafb87bd84a3486018269f8791cbc in term 1.
I20260805 23:20:04.216634 15103 leader_election.cc:302] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97e957c4f0e241f58d2727b50403e3f8, eb9dafb87bd84a3486018269f8791cbc; no voters:
I20260805 23:20:04.216904 15312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: Bootstrap starting.
W20260805 23:20:04.218614 15346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:20:04.218686 15346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:04.216012 15083 master_runner.cc:186] Error getting master registration for 127.23.88.124:39037: OK, instance_id { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" instance_seqno: 1785972004147940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:20:04.220036 15083 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:20:04.220162 15321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:20:04.220248 15312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:04.220428 15321 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [term 1 LEADER]: Becoming Leader. State: Replica: eb9dafb87bd84a3486018269f8791cbc, State: Running, Role: LEADER
I20260805 23:20:04.220786 15321 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.222774 15321 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9dafb87bd84a3486018269f8791cbc. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.222924 15321 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:04.220561 15163 master_runner.cc:186] Error getting master registration for 127.23.88.124:39037: OK, instance_id { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" instance_seqno: 1785972004147940 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:20:04.224164 15163 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:20:04.225517 15354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:20:04.226070 15312 log.cc:824] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:04.226589 15354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:20:04.229910 15312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: No bootstrap required, opened a new log
I20260805 23:20:04.236929 15312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.237550 15312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9de512fef63b4dde8b08ee597535da0e, State: Initialized, Role: FOLLOWER
I20260805 23:20:04.238205 15312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } }
I20260805 23:20:04.238962 15159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.224883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.23.88.126:37647)
0805 23:20:04.225000 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.225005 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.229447 (+ 4442us) server_negotiation.cc:407] Connection header received
0805 23:20:04.229627 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.229630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.229682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.229734 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.230785 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.231634 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.232893 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.233239 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.233317 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.235308 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.235312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.235315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.235346 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.237083 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.237580 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.237583 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.237585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.237669 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.237974 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.237977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.237979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.238197 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.238296 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.238647 (+ 351us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.238788 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:20:04.241120 15285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eb9dafb87bd84a3486018269f8791cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9de512fef63b4dde8b08ee597535da0e"
I20260805 23:20:04.241379 15285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:04.243530 15285 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb9dafb87bd84a3486018269f8791cbc in term 1.
I20260805 23:20:04.243902 15312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:04.244724 15285 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [term 1 FOLLOWER]: Refusing update from remote peer eb9dafb87bd84a3486018269f8791cbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:20:04.245397 15352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:20:04.247185 15206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [term 1 FOLLOWER]: Refusing update from remote peer eb9dafb87bd84a3486018269f8791cbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:20:04.247953 15352 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:20:04.252930 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9dafb87bd84a3486018269f8791cbc. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.250959 15373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9dafb87bd84a3486018269f8791cbc. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.253213 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.253265 15373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.251405 15360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.253419 15360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.257041 15352 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.257191 15352 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:04.257553 15321 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.257663 15321 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:04.257781 15359 mvcc.cc:205] Tried to move back new op lower bound from 7315341329376645120 to 7315341329292292096. Current Snapshot: MvccSnapshot[applied={T|T < 7315341329376645120}]
I20260805 23:20:04.259011 15334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.259166 15334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.259415 15360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb9dafb87bd84a3486018269f8791cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb9dafb87bd84a3486018269f8791cbc" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 37647 } } peers { permanent_uuid: "97e957c4f0e241f58d2727b50403e3f8" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 34451 } } peers { permanent_uuid: "9de512fef63b4dde8b08ee597535da0e" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 39037 } } }
I20260805 23:20:04.259546 15360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:04.262005 15376 mvcc.cc:205] Tried to move back new op lower bound from 7315341329376645120 to 7315341329292292096. Current Snapshot: MvccSnapshot[applied={T|T < 7315341329376645120}]
I20260805 23:20:04.262094 15317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.223549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51232 (local address 127.23.88.124:39037)
0805 23:20:04.223705 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.223710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.226759 (+ 3049us) server_negotiation.cc:407] Connection header received
0805 23:20:04.227031 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.227036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.227255 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.227346 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.239828 (+ 12482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.240582 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.243127 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.243517 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.243603 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.254959 (+ 11356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.254964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.254967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.255014 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.256969 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.260447 (+ 3478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.260452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.260455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.260558 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.260916 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.260919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.260921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.261243 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.261392 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.261784 (+ 392us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.261934 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260805 23:20:04.262781 15238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:04.225096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.23.88.125:34451)
0805 23:20:04.225217 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.225224 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.239714 (+ 14490us) server_negotiation.cc:407] Connection header received
0805 23:20:04.241956 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.241962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.242036 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.242101 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.244388 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.245176 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.251929 (+ 6753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:04.252306 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:04.252403 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:04.258506 (+ 6103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:04.258510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:04.258512 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:04.258544 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:04.260596 (+ 2052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.261110 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.261113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.261115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.261201 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:04.261524 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:04.261527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:04.261529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:04.261772 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:04.261897 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:04.262509 (+ 612us) server_negotiation.cc:299] Negotiation successful
0805 23:20:04.262642 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260805 23:20:04.268533 15354 catalog_manager.cc:1347] Generated new cluster ID: 6c6434619f604d998d7273650388622d
I20260805 23:20:04.268635 15354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:20:04.273883 15372 catalog_manager.cc:1259] Loaded cluster ID: 6c6434619f604d998d7273650388622d
I20260805 23:20:04.274065 15372 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: loading cluster ID for follower catalog manager: success
W20260805 23:20:04.274878 15372 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260805 23:20:04.275058 15372 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:04.306814 15354 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:20:04.308319 15354 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:20:04.334712 15354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9dafb87bd84a3486018269f8791cbc: Generated new TSK 0
I20260805 23:20:04.336177 15354 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:20:04.371236 15317 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.367417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.23.88.124:39037)
0805 23:20:04.367604 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.367608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.369913 (+ 2305us) server_negotiation.cc:407] Connection header received
0805 23:20:04.370132 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.370136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.370194 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.370250 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.371018 (+ 768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51236: BlockingRecv error: recv got EOF from 127.0.0.1:51236 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260805 23:20:04.373447 15159 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.368445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39464 (local address 127.23.88.126:37647)
0805 23:20:04.368581 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.368586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.371800 (+ 3214us) server_negotiation.cc:407] Connection header received
0805 23:20:04.372506 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.372510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.372570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.372628 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.373283 (+ 655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39464: BlockingRecv error: recv got EOF from 127.0.0.1:39464 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260805 23:20:04.374702 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.369238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.23.88.125:34451)
0805 23:20:04.369373 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.369378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.372191 (+ 2813us) server_negotiation.cc:407] Connection header received
0805 23:20:04.373814 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.373824 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.373909 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.373969 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.374566 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36372: BlockingRecv error: recv got EOF from 127.0.0.1:36372 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260805 23:20:04.381690 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.379524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.23.88.125:34451)
0805 23:20:04.379666 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.379671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.380265 (+ 594us) server_negotiation.cc:407] Connection header received
0805 23:20:04.380624 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.380629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.380735 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.380801 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.381538 (+ 737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36378: BlockingRecv error: recv got EOF from 127.0.0.1:36378 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260805 23:20:04.388325 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.386186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36394 (local address 127.23.88.125:34451)
0805 23:20:04.386334 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.386339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.386989 (+ 650us) server_negotiation.cc:407] Connection header received
0805 23:20:04.387409 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.387414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.387479 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.387539 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.388195 (+ 656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36394: BlockingRecv error: recv got EOF from 127.0.0.1:36394 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260805 23:20:04.395440 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.393557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.23.88.125:34451)
0805 23:20:04.393690 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.393695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.394312 (+ 617us) server_negotiation.cc:407] Connection header received
0805 23:20:04.394537 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.394541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.394603 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.394661 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.395307 (+ 646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36408: BlockingRecv error: recv got EOF from 127.0.0.1:36408 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260805 23:20:04.404799 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.402571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.23.88.125:34451)
0805 23:20:04.402761 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.402767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.403503 (+ 736us) server_negotiation.cc:407] Connection header received
0805 23:20:04.403851 (+ 348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.403855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.403921 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.403985 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.404637 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36414: BlockingRecv error: recv got EOF from 127.0.0.1:36414 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260805 23:20:04.418882 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.416407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36430 (local address 127.23.88.125:34451)
0805 23:20:04.416563 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.416569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.417377 (+ 808us) server_negotiation.cc:407] Connection header received
0805 23:20:04.417643 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.417647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.417717 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.417961 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.418725 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36430: BlockingRecv error: recv got EOF from 127.0.0.1:36430 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260805 23:20:04.440652 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.438258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36436 (local address 127.23.88.125:34451)
0805 23:20:04.438433 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.438439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.439159 (+ 720us) server_negotiation.cc:407] Connection header received
0805 23:20:04.439429 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.439434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.439498 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.439555 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.440520 (+ 965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36436: BlockingRecv error: recv got EOF from 127.0.0.1:36436 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260805 23:20:04.476866 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.475357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36452 (local address 127.23.88.125:34451)
0805 23:20:04.475516 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.475521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.475829 (+ 308us) server_negotiation.cc:407] Connection header received
0805 23:20:04.475960 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.475963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.476019 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.476111 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.476711 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36452: BlockingRecv error: recv got EOF from 127.0.0.1:36452 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260805 23:20:04.544349 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.543160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.23.88.125:34451)
0805 23:20:04.543340 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.543344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.543422 (+ 78us) server_negotiation.cc:407] Connection header received
0805 23:20:04.543558 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.543560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.543611 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.543697 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.544196 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36466: BlockingRecv error: recv got EOF from 127.0.0.1:36466 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260805 23:20:04.675700 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.674536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36476 (local address 127.23.88.125:34451)
0805 23:20:04.674701 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.674708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.674855 (+ 147us) server_negotiation.cc:407] Connection header received
0805 23:20:04.674968 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.674971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.675028 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.675123 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.675588 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36476: BlockingRecv error: recv got EOF from 127.0.0.1:36476 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260805 23:20:04.935020 15238 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:04.933964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.23.88.125:34451)
0805 23:20:04.934102 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:04.934107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:04.934178 (+ 71us) server_negotiation.cc:407] Connection header received
0805 23:20:04.934303 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:04.934306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:04.934360 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:04.934446 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:04.934907 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36486: BlockingRecv error: recv got EOF from 127.0.0.1:36486 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260805 23:20:05.220166 15346 catalog_manager.cc:1259] Loaded cluster ID: 6c6434619f604d998d7273650388622d
I20260805 23:20:05.220288 15346 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: loading cluster ID for follower catalog manager: success
I20260805 23:20:05.222110 15346 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: acquiring CA information for follower catalog manager: success
I20260805 23:20:05.223418 15346 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 97e957c4f0e241f58d2727b50403e3f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:20:05.277849 15372 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: acquiring CA information for follower catalog manager: success
I20260805 23:20:05.279347 15372 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9de512fef63b4dde8b08ee597535da0e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:20:05.453333 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:05.449196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.23.88.125:34451)
0805 23:20:05.449471 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:05.449476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:05.449494 (+ 18us) server_negotiation.cc:407] Connection header received
0805 23:20:05.449618 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:05.449620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:05.449749 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:05.449866 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:05.450206 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.450798 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.451751 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.452018 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.452132 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:05.452456 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:05.452963 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:05.453077 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:05.453138 (+ 61us) server_negotiation.cc:299] Negotiation successful
0805 23:20:05.453194 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":112,"threads_started":1}
I20260805 23:20:05.460736 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:05.456944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.23.88.124:39037)
0805 23:20:05.457182 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:05.457185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:05.457201 (+ 16us) server_negotiation.cc:407] Connection header received
0805 23:20:05.457340 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:05.457343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:05.457469 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:05.457581 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:05.457765 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.458302 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.459173 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.459451 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.459541 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:05.459874 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:05.460356 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:05.460463 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:05.460550 (+ 87us) server_negotiation.cc:299] Negotiation successful
0805 23:20:05.460606 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260805 23:20:05.468552 15453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:05.464008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.23.88.125:34451)
0805 23:20:05.464282 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:05.464287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:05.464588 (+ 301us) server_negotiation.cc:407] Connection header received
0805 23:20:05.464844 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:05.464862 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:05.464931 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:05.465027 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:05.465352 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.465949 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467140 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.467481 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467604 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:05.467965 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:05.468067 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:05.468139 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:05.468324 (+ 185us) server_negotiation.cc:299] Negotiation successful
0805 23:20:05.468387 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260805 23:20:05.468560 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:05.464120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51240 (local address 127.23.88.124:39037)
0805 23:20:05.464274 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:05.464279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:05.464577 (+ 298us) server_negotiation.cc:407] Connection header received
0805 23:20:05.464822 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:05.464828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:05.464892 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:05.464945 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:05.465407 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.466233 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467217 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.467641 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467756 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:05.468069 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:05.468152 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:05.468196 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:05.468371 (+ 175us) server_negotiation.cc:299] Negotiation successful
0805 23:20:05.468431 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260805 23:20:05.469139 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:05.464084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.23.88.126:37647)
0805 23:20:05.464600 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:05.464604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:05.464718 (+ 114us) server_negotiation.cc:407] Connection header received
0805 23:20:05.464962 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:05.464965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:05.465149 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:05.465289 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:05.465523 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.466316 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467565 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:05.467889 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:05.467991 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:05.468338 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:05.468710 (+ 372us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:05.468800 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:05.468934 (+ 134us) server_negotiation.cc:299] Negotiation successful
0805 23:20:05.469002 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":76,"threads_started":1}
I20260805 23:20:05.472556 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15083
I20260805 23:20:05.480528 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15163
I20260805 23:20:05.488186 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15240
2026-08-05T23:20:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3254 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 05 23:20:05 dist-test-slave-vkjp krb5kdc[15470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 05 23:20:05 dist-test-slave-vkjp krb5kdc[15470](info): set up 2 sockets
Aug 05 23:20:05 dist-test-slave-vkjp krb5kdc[15470](info): commencing operation
krb5kdc: starting...
W20260805 23:20:06.578176 23905 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/test-user.keytab.
Aug 05 23:20:06 dist-test-slave-vkjp krb5kdc[15470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-05T23:20:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-05T23:20:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.88.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
Entry for principal kudu/127.23.88.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab.
I20260805 23:20:06.720278 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34669
--webserver_interface=127.23.88.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260805 23:20:06.842545 15485 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:06.842895 15485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:06.842921 15485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:06.842960 15485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:06.847271 15485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:06.847355 15485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:06.847375 15485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:06.847393 15485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:06.847455 15485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:06.853637 15485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.126.keytab
--principal=kudu/127.23.88.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.126:34669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.88.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15485
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:06.855051 15485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:06.856298 15485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:06.865971 15490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:06.865971 15491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:06.866123 15493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:06.866405 15485 server_base.cc:1034] running on GCE node
I20260805 23:20:06.867194 15485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:06.868604 15485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:06.869987 15485 hybrid_clock.cc:648] HybridClock initialized: now 1785972006869964 us; error 35 us; skew 500 ppm
Aug 05 23:20:06 dist-test-slave-vkjp krb5kdc[15470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:06.878575 15485 init.cc:377] Logged in from keytab as kudu/127.23.88.126@KRBTEST.COM (short username kudu)
I20260805 23:20:06.879906 15485 webserver.cc:460] Webserver started at http://127.23.88.126:44007/ using document root <none> and password file <none>
I20260805 23:20:06.880578 15485 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:06.880689 15485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:06.880945 15485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:06.883451 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data/instance:
uuid: "98f67b78652e49499a20f0cda78dfbf4"
format_stamp: "Formatted at 2026-08-05 23:20:06 on dist-test-slave-vkjp"
server_key: "47463283ac9dfb67772f0920cc60410f"
server_key_iv: "3d41fb14ebc547e93ac224d321b90f82"
server_key_version: "encryptionkey@0"
I20260805 23:20:06.884125 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance:
uuid: "98f67b78652e49499a20f0cda78dfbf4"
format_stamp: "Formatted at 2026-08-05 23:20:06 on dist-test-slave-vkjp"
server_key: "47463283ac9dfb67772f0920cc60410f"
server_key_iv: "3d41fb14ebc547e93ac224d321b90f82"
server_key_version: "encryptionkey@0"
I20260805 23:20:06.888435 15485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:20:06.891211 15500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:06.892590 15485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260805 23:20:06.892747 15485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal
uuid: "98f67b78652e49499a20f0cda78dfbf4"
format_stamp: "Formatted at 2026-08-05 23:20:06 on dist-test-slave-vkjp"
server_key: "47463283ac9dfb67772f0920cc60410f"
server_key_iv: "3d41fb14ebc547e93ac224d321b90f82"
server_key_version: "encryptionkey@0"
I20260805 23:20:06.892881 15485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:06.918025 15485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:06.921947 15485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:06.922227 15485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:06.931290 15485 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.126:34669
I20260805 23:20:06.931603 15553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.126:34669 every 8 connection(s)
I20260805 23:20:06.932543 15485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/data/info.pb
I20260805 23:20:06.935549 15554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:06.938835 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15485
I20260805 23:20:06.938995 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-0/wal/instance
I20260805 23:20:06.939357 23905 external_mini_cluster.cc:1428] Setting key 6d6c18a986b7d14d5d05230ae64a6b25
I20260805 23:20:06.940066 15554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.88.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.125@KRBTEST.COM" created.
Aug 05 23:20:06 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:20:06.958968 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:06.943075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.23.88.126:34669)
0805 23:20:06.943607 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:06.943628 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:06.943899 (+ 271us) server_negotiation.cc:407] Connection header received
0805 23:20:06.945060 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:06.945064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:06.945264 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:06.945459 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:06.946330 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:06.947304 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:06.948343 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:06.948810 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:06.948994 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:06.953116 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:06.953135 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:06.953155 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:06.953196 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:06.955526 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:06.956101 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:06.956107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:06.956109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:06.956232 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:06.956571 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:06.956574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:06.956576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:06.957012 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:06.957168 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:06.957534 (+ 366us) server_negotiation.cc:299] Negotiation successful
0805 23:20:06.957818 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":150,"threads_started":1}
I20260805 23:20:06.958968 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:06.943106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34669 (local address 127.0.0.1:57012)
0805 23:20:06.943748 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:06.943786 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:06.944707 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:06.945454 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:06.945460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:06.945927 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:06.946176 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:06.946190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:06.947447 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:06.947450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:06.948217 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:06.948227 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:06.948992 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:06.948996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:06.949109 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:06.949808 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:06.949825 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:06.952873 (+ 3048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:06.955689 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:06.955699 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:06.955711 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:06.955975 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:06.956333 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:06.956337 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:06.956339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:06.956466 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:06.957134 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:06.957141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:06.957290 (+ 149us) client_negotiation.cc:769] Sending connection context
0805 23:20:06.957472 (+ 182us) client_negotiation.cc:240] Negotiation successful
0805 23:20:06.957869 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":108,"thread_start_us":167,"threads_started":1}
I20260805 23:20:06.962381 15554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:06.963541 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:06.963043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42682)
0805 23:20:06.963292 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:06.963381 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260805 23:20:06.963768 15503 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
I20260805 23:20:06.964838 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:06.964583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42694)
0805 23:20:06.964719 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:06.964771 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260805 23:20:06.965113 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:41935: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
Entry for principal kudu/127.23.88.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab.
I20260805 23:20:06.970880 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:41935
--webserver_interface=127.23.88.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:20:07.014444 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } attempt: 1
I20260805 23:20:07.015688 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.015226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42702)
0805 23:20:07.015480 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.015572 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260805 23:20:07.016573 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.016326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42704)
0805 23:20:07.016445 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.016503 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260805 23:20:07.016724 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:41935: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
I20260805 23:20:07.084955 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } attempt: 2
I20260805 23:20:07.085922 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.085599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42714)
0805 23:20:07.085773 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.085842 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260805 23:20:07.086778 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.086546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42720)
0805 23:20:07.086645 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.086692 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260805 23:20:07.087024 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.125:41935: Network error: Client connection negotiation failed: client connection to 127.23.88.125:41935: connect: Connection refused (error 111)
W20260805 23:20:07.093494 15563 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:07.093880 15563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:07.093945 15563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:07.094025 15563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:07.098590 15563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:07.098683 15563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:07.098721 15563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:07.098778 15563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:07.098814 15563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:07.104949 15563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.125.keytab
--principal=kudu/127.23.88.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.125:41935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.88.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:07.106345 15563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:07.107769 15563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:07.117606 15569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:07.117606 15571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:07.117607 15568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:07.118022 15563 server_base.cc:1034] running on GCE node
I20260805 23:20:07.118705 15563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:07.119998 15563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:07.121203 15563 hybrid_clock.cc:648] HybridClock initialized: now 1785972007121192 us; error 35 us; skew 500 ppm
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:07.124321 15563 init.cc:377] Logged in from keytab as kudu/127.23.88.125@KRBTEST.COM (short username kudu)
I20260805 23:20:07.125296 15563 webserver.cc:460] Webserver started at http://127.23.88.125:43307/ using document root <none> and password file <none>
I20260805 23:20:07.125831 15563 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:07.125893 15563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:07.126077 15563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:07.128268 15563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data/instance:
uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "ffa655efccefb3b9654ae96a070c5a08"
server_key_iv: "5378adec7d9a9474695897a8c7184f60"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.128810 15563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal/instance:
uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "ffa655efccefb3b9654ae96a070c5a08"
server_key_iv: "5378adec7d9a9474695897a8c7184f60"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.132645 15563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260805 23:20:07.135244 15578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:07.136574 15563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:20:07.136724 15563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal
uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "ffa655efccefb3b9654ae96a070c5a08"
server_key_iv: "5378adec7d9a9474695897a8c7184f60"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.136840 15563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:07.156131 15563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:07.159788 15563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:07.160028 15563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:07.167567 15563 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.125:41935
I20260805 23:20:07.167915 15633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.125:41935 every 8 connection(s)
I20260805 23:20:07.168625 15563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/data/info.pb
I20260805 23:20:07.171657 15634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:07.175683 15634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.178292 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } attempt: 3
I20260805 23:20:07.178671 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15563
I20260805 23:20:07.178802 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-1/wal/instance
I20260805 23:20:07.179147 23905 external_mini_cluster.cc:1428] Setting key d58c7fc5e6c599934f60c3402d267022
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.88.124@KRBTEST.COM; defaulting to no policy
I20260805 23:20:07.192549 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.178863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42730)
0805 23:20:07.179040 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.179054 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.179196 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.180794 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.180799 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.181083 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.181243 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.181253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.183038 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.183045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.183637 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.183645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.184450 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.184457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.184689 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.185348 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.185365 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.187191 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.190123 (+ 2932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.190126 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.190127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.190366 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.190833 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.190837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.190839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.190958 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.191919 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.191922 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.192054 (+ 132us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.192165 (+ 111us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.192339 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":26}
I20260805 23:20:07.192878 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.177880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.23.88.126:34669)
0805 23:20:07.178075 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.178081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.179178 (+ 1097us) server_negotiation.cc:407] Connection header received
0805 23:20:07.180525 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.180531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.180605 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.180881 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.182111 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.182664 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.184316 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.184793 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.184923 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.188235 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.188239 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.188242 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.188277 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.190671 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.191336 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.191339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.191341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.191428 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.191725 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.191728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.191730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.191967 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.192062 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.192549 (+ 487us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.192723 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":70}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.88.124@KRBTEST.COM" created.
I20260805 23:20:07.193775 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.179422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42730 (local address 127.23.88.125:41935)
0805 23:20:07.179753 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.179764 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.179821 (+ 57us) server_negotiation.cc:407] Connection header received
0805 23:20:07.180250 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.180274 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.180552 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.181443 (+ 891us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.181826 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.182885 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.183763 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.184302 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.184471 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.187353 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.187361 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.187378 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.187411 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.189708 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.190519 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.190528 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.190538 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.190703 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.191130 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.191133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.191135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.191791 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.191945 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.192353 (+ 408us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.192722 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":223,"spinlock_wait_cycles":7808,"thread_start_us":112,"threads_started":1}
I20260805 23:20:07.193781 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.178297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34669 (local address 127.0.0.1:57016)
0805 23:20:07.178992 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.179063 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.180246 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.181069 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.181077 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.181431 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.181933 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.181944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.182794 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.182801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.184164 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.184179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.184974 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.184979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.185084 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.185642 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.185663 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.188050 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.190833 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.190844 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.190851 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.191212 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.191541 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.191544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.191545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.191624 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.192080 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.192088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.192261 (+ 173us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.192433 (+ 172us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.192903 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":223,"spinlock_wait_cycles":17024,"thread_start_us":214,"threads_started":1}
I20260805 23:20:07.196193 15634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.196187 15554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.197319 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.196804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49462)
0805 23:20:07.197159 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.197229 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
I20260805 23:20:07.198213 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.197881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49464)
0805 23:20:07.198089 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.198142 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260805 23:20:07.198354 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:07.206749 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.196843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42746)
0805 23:20:07.197004 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.197022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.197121 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.197417 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.197423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.197656 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.197837 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.197848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.198856 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.198860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.199852 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.199868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.200384 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.200390 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.200546 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.201096 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.201113 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.202883 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.204947 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.204950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.204953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.205180 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.205671 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.205675 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.205677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.205760 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.206169 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.206172 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.206265 (+ 93us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.206374 (+ 109us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.206545 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260805 23:20:07.206749 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.197042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.23.88.125:41935)
0805 23:20:07.197160 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.197166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.197192 (+ 26us) server_negotiation.cc:407] Connection header received
0805 23:20:07.197233 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.197235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.197283 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.197380 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.197967 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.198725 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.200002 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.200245 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.200349 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.203060 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.203066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.203069 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.203146 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.204791 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.205451 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.205454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.205455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.205548 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.205862 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.205865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.205869 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.206067 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.206181 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.206429 (+ 248us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.206607 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":49}
I20260805 23:20:07.208086 15634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.209009 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.208650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49466)
0805 23:20:07.208828 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.208927 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260805 23:20:07.209923 15581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
I20260805 23:20:07.211028 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.210772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49470)
0805 23:20:07.210883 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.210932 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260805 23:20:07.211272 15634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
Entry for principal kudu/127.23.88.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab.
I20260805 23:20:07.215211 23905 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
/tmp/dist-test-taskRmBmWz/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:33309
--webserver_interface=127.23.88.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.88.84:39045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260805 23:20:07.229516 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 1
I20260805 23:20:07.230659 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.230283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49482)
0805 23:20:07.230509 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.230577 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260805 23:20:07.231748 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.231463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49496)
0805 23:20:07.231599 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.231663 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260805 23:20:07.231937 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
I20260805 23:20:07.260594 15634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 1
I20260805 23:20:07.261597 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.261272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49498)
0805 23:20:07.261444 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.261512 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260805 23:20:07.262440 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.262189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49514)
0805 23:20:07.262311 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.262369 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260805 23:20:07.262607 15634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
I20260805 23:20:07.307200 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 2
I20260805 23:20:07.308218 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.307825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49524)
0805 23:20:07.308032 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.308111 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260805 23:20:07.309115 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.308830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49536)
0805 23:20:07.308977 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.309043 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260805 23:20:07.309289 15554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
I20260805 23:20:07.330802 15634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 2
I20260805 23:20:07.331800 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.331438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49540)
0805 23:20:07.331622 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.331699 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260805 23:20:07.332772 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.332488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49556)
0805 23:20:07.332615 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.332684 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260805 23:20:07.332958 15634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.88.124:33309: Network error: Client connection negotiation failed: client connection to 127.23.88.124:33309: connect: Connection refused (error 111)
W20260805 23:20:07.335819 15641 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260805 23:20:07.336187 15641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260805 23:20:07.336253 15641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260805 23:20:07.336334 15641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260805 23:20:07.340592 15641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260805 23:20:07.340685 15641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260805 23:20:07.340721 15641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260805 23:20:07.340780 15641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260805 23:20:07.340817 15641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260805 23:20:07.346504 15641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.88.84:39045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.88.126:34669,127.23.88.125:41935,127.23.88.124:33309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/krb5kdc/kudu_127.23.88.124.keytab
--principal=kudu/127.23.88.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.88.124:33309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.88.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 05 Aug 2026 22:32:10 UTC on ccycloud.quasar-zhglpi.root.comops.site
build id 4569239
I20260805 23:20:07.347743 15641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260805 23:20:07.348798 15641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260805 23:20:07.357827 15647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:07.357831 15649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260805 23:20:07.357826 15646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260805 23:20:07.358001 15641 server_base.cc:1034] running on GCE node
I20260805 23:20:07.358667 15641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260805 23:20:07.359939 15641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260805 23:20:07.361132 15641 hybrid_clock.cc:648] HybridClock initialized: now 1785972007361091 us; error 55 us; skew 500 ppm
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260805 23:20:07.363965 15641 init.cc:377] Logged in from keytab as kudu/127.23.88.124@KRBTEST.COM (short username kudu)
I20260805 23:20:07.364933 15641 webserver.cc:460] Webserver started at http://127.23.88.124:36445/ using document root <none> and password file <none>
I20260805 23:20:07.365518 15641 fs_manager.cc:359] Metadata directory not provided
I20260805 23:20:07.365614 15641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260805 23:20:07.365851 15641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260805 23:20:07.368026 15641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data/instance:
uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "6093f834955928f02c94d76b2ee217fe"
server_key_iv: "68be495f145ade655457745a7a2cdf2e"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.368639 15641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal/instance:
uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "6093f834955928f02c94d76b2ee217fe"
server_key_iv: "68be495f145ade655457745a7a2cdf2e"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.372606 15641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260805 23:20:07.375280 15656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260805 23:20:07.376628 15641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260805 23:20:07.376785 15641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data,/tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal
uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9"
format_stamp: "Formatted at 2026-08-05 23:20:07 on dist-test-slave-vkjp"
server_key: "6093f834955928f02c94d76b2ee217fe"
server_key_iv: "68be495f145ade655457745a7a2cdf2e"
server_key_version: "encryptionkey@0"
I20260805 23:20:07.376912 15641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260805 23:20:07.396335 15641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260805 23:20:07.400120 15641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260805 23:20:07.400318 15641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260805 23:20:07.407869 15641 rpc_server.cc:307] RPC server started. Bound to: 127.23.88.124:33309
I20260805 23:20:07.408205 15711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.88.124:33309 every 8 connection(s)
I20260805 23:20:07.408506 15554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 3
I20260805 23:20:07.409031 15641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/data/info.pb
I20260805 23:20:07.412588 23905 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu as pid 15641
I20260805 23:20:07.412736 23905 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRmBmWz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785971715702001-23905-0/minicluster-data/master-2/wal/instance
I20260805 23:20:07.412786 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260805 23:20:07.413093 23905 external_mini_cluster.cc:1428] Setting key 4ab9d21ebf7302da06befd4104c83dd4
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.126@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:07.417814 15712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } has no permanent_uuid. Determining permanent_uuid...
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:07.424131 15634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } attempt: 3
I20260805 23:20:07.424408 15560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.409088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49572)
0805 23:20:07.409219 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.409237 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.409337 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.412192 (+ 2855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.412194 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.412461 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.412615 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.412624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.414381 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.414387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.415092 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.415122 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.416117 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.416122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.416216 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.416824 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.416841 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.418583 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.422150 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.422154 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.422157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.422446 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.422820 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.422826 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.422830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.422939 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.423746 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.423749 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.423848 (+ 99us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.423951 (+ 103us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.424227 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260805 23:20:07.426858 15716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.413776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.23.88.124:33309)
0805 23:20:07.414548 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.414556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.414569 (+ 13us) server_negotiation.cc:407] Connection header received
0805 23:20:07.414602 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.414604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.414646 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.414737 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.415798 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.416504 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.417731 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.417962 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.418072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.421523 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.421527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.421531 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.421559 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.423484 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.423904 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.423909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.423910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.423995 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.424316 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.424319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.424321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.425229 (+ 908us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.425441 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.426240 (+ 799us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.426351 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":703,"thread_start_us":114,"threads_started":1}
I20260805 23:20:07.426884 15714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.410021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.23.88.124:33309)
0805 23:20:07.410479 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.410499 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.410546 (+ 47us) server_negotiation.cc:407] Connection header received
0805 23:20:07.411420 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.411442 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.411865 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.412218 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.412821 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.414240 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.415236 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.415975 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.416158 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.418810 (+ 2652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.418816 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.418829 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.418857 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.422015 (+ 3158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.422575 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.422580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.422586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.422715 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.423060 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.423063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.423065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.423613 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.425240 (+ 1627us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.425410 (+ 170us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.425767 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.125@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:07.432734 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.419948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57028 (local address 127.23.88.126:34669)
0805 23:20:07.420095 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.420101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.420615 (+ 514us) server_negotiation.cc:407] Connection header received
0805 23:20:07.420999 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.421003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.421072 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.421130 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.422192 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.422851 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.424225 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.424579 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.424659 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.428150 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.428153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.428155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.428183 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.429963 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.430520 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.430526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.430531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.430627 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.431215 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.431219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.431222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.431564 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.431679 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.432309 (+ 630us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.432473 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260805 23:20:07.433207 15554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: Bootstrap starting.
I20260805 23:20:07.433423 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.419893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.126:34669 (local address 127.0.0.1:57028)
0805 23:20:07.420480 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.420534 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.420807 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.421317 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.421323 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.421930 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.422086 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.422097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.422970 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.422975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.424050 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.424065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.424947 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.424951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.425253 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.425790 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.425803 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.427993 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.430095 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.430105 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.430110 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.430395 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.430904 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.430909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.430912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.431044 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.431798 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.431808 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.431972 (+ 164us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.432913 (+ 941us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.433158 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260805 23:20:07.434605 15712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.436340 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.424598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49588)
0805 23:20:07.424857 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.424882 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.424968 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.425576 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.425580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.425806 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.425934 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.425942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.427075 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.427101 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.427840 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.427850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.429375 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.429378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.429488 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.429970 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.429988 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.431527 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.434539 (+ 3012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.434543 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.434545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.434818 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.435199 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.435202 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.435204 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.435303 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.435808 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.435812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.435926 (+ 114us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.436021 (+ 95us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.436176 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260805 23:20:07.436996 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.425005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.23.88.124:33309)
0805 23:20:07.425326 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.425335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.425347 (+ 12us) server_negotiation.cc:407] Connection header received
0805 23:20:07.425381 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.425384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.425425 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.426070 (+ 645us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.426190 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.426912 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.428911 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.429236 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.429705 (+ 469us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.431699 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.431703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.431706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.431730 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.434380 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.434956 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.434960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.434964 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.435061 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.435440 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.435443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.435446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.435684 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.436642 (+ 958us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.436717 (+ 75us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.436831 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":130,"threads_started":1}
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.126@KRBTEST.COM
I20260805 23:20:07.438159 15554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:07.439034 15554 log.cc:824] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:07.440431 15634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: Bootstrap starting.
I20260805 23:20:07.441569 15554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: No bootstrap required, opened a new log
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:07.442991 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.429820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.23.88.126:34669)
0805 23:20:07.430671 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.430773 (+ 102us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.430806 (+ 33us) server_negotiation.cc:407] Connection header received
0805 23:20:07.430861 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.430865 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.430929 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.431034 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.432656 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.433526 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.434833 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.435240 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.435374 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.438465 (+ 3091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.438469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.438472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.438499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.440366 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.440856 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.440859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.440861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.440953 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.441310 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.441316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.441318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.441580 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.441829 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.442639 (+ 810us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.442773 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":772,"thread_start_us":97,"threads_started":1}
I20260805 23:20:07.443555 15634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:07.444504 15634 log.cc:824] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:07.445883 15554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.446235 15554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f67b78652e49499a20f0cda78dfbf4, State: Initialized, Role: FOLLOWER
I20260805 23:20:07.446653 15554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.446928 15634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: No bootstrap required, opened a new log
I20260805 23:20:07.446980 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.435443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42748 (local address 127.23.88.125:41935)
0805 23:20:07.435571 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.435575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.435650 (+ 75us) server_negotiation.cc:407] Connection header received
0805 23:20:07.435833 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.435836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.435898 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.435939 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.436562 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.437364 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.440351 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.440742 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.440829 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.443276 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.443280 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.443282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.443309 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.445104 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.445648 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.445651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.445653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.445732 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.446024 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.446027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.446029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.446265 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.446350 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.446716 (+ 366us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.446834 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260805 23:20:07.447436 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.435166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.125:41935 (local address 127.0.0.1:42748)
0805 23:20:07.435583 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.435598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.435717 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.436108 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.436111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.436299 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.436410 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.436419 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.437509 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.437514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.440192 (+ 2678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.440207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.441034 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.441040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.441141 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.441614 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.441634 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.443120 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.445248 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.445254 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.445259 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.445528 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.445840 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.445842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.445844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.445924 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.446414 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.446417 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.446524 (+ 107us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.447124 (+ 600us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.447280 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260805 23:20:07.448252 15554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:07.448869 15712 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } has no permanent_uuid. Determining permanent_uuid...
I20260805 23:20:07.451149 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.451406 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: This master's current role is: FOLLOWER
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.88.125@KRBTEST.COM
I20260805 23:20:07.452343 15634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.452574 15634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f7a7d8e8c8544e8bcbe04c906d36b99, State: Initialized, Role: FOLLOWER
I20260805 23:20:07.452899 15634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.453481 15735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.453679 15735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.454519 15634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: configured and running, proceeding with master startup.
W20260805 23:20:07.456789 15734 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:20:07.456941 15734 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:07.454361 15485 master_runner.cc:186] Error getting master registration for 127.23.88.125:41935: OK, instance_id { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" instance_seqno: 1785972007165284 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:20:07.458768 15485 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:20:07.457784 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.444428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.23.88.125:41935)
0805 23:20:07.445174 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.445182 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.445196 (+ 14us) server_negotiation.cc:407] Connection header received
0805 23:20:07.445230 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.445233 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.445277 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.446139 (+ 862us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.446259 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.447163 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.448354 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.449463 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.449579 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.452759 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.452763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.452766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.452798 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.454811 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.455542 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.455547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.455550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.455651 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.455959 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.455962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.455964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.456189 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.456404 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.457103 (+ 699us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.457236 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":116,"threads_started":1}
Aug 05 23:20:07 dist-test-slave-vkjp krb5kdc[15470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785972007, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.88.124@KRBTEST.COM for kudu/127.23.88.124@KRBTEST.COM
I20260805 23:20:07.463514 23905 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260805 23:20:07.463912 15747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:20:07.464030 15747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:07.465442 15528 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "98f67b78652e49499a20f0cda78dfbf4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57044
I20260805 23:20:07.465777 15528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260805 23:20:07.465931 15528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:07.467061 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.449726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.23.88.124:33309)
0805 23:20:07.449836 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.449840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.449873 (+ 33us) server_negotiation.cc:407] Connection header received
0805 23:20:07.449908 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.449912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.449952 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.450035 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.457430 (+ 7395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.458288 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.459304 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.460017 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.460099 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.462724 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.462728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.462730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.462755 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.464394 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.465353 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.465358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.465360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.465481 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.466047 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.466051 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.466053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.466320 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.466406 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.466730 (+ 324us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.466883 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260805 23:20:07.467609 15528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.467921 15717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.449366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.88.124:33309 (local address 127.0.0.1:49590)
0805 23:20:07.449637 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0805 23:20:07.449651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0805 23:20:07.449722 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0805 23:20:07.456974 (+ 7252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0805 23:20:07.456980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0805 23:20:07.457184 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0805 23:20:07.457285 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.457292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.458418 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.458424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.459174 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0805 23:20:07.459188 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.460262 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.460268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0805 23:20:07.460345 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.460902 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0805 23:20:07.460918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0805 23:20:07.462564 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0805 23:20:07.464509 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.464515 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.464518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.465072 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.465601 (+ 529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0805 23:20:07.465604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0805 23:20:07.465606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0805 23:20:07.465782 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0805 23:20:07.466468 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0805 23:20:07.466470 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0805 23:20:07.466559 (+ 89us) client_negotiation.cc:769] Sending connection context
0805 23:20:07.467552 (+ 993us) client_negotiation.cc:240] Negotiation successful
0805 23:20:07.467706 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204}
I20260805 23:20:07.468899 15528 leader_election.cc:288] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [CANDIDATE]: Term 1 election: Requested vote from peers 3f7a7d8e8c8544e8bcbe04c906d36b99 (127.23.88.125:41935), 5e83cc2fe4c54ef5a4bcb753af000ae9 (127.23.88.124:33309)
I20260805 23:20:07.469988 15607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98f67b78652e49499a20f0cda78dfbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99"
I20260805 23:20:07.470263 15607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:07.471197 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: Bootstrap starting.
I20260805 23:20:07.467854 15563 master_runner.cc:186] Error getting master registration for 127.23.88.124:33309: OK, instance_id { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" instance_seqno: 1785972007405724 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260805 23:20:07.471736 15607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f67b78652e49499a20f0cda78dfbf4 in term 1.
I20260805 23:20:07.471848 15563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260805 23:20:07.472191 15503 leader_election.cc:302] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f7a7d8e8c8544e8bcbe04c906d36b99, 98f67b78652e49499a20f0cda78dfbf4; no voters:
I20260805 23:20:07.472528 15723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260805 23:20:07.472807 15723 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [term 1 LEADER]: Becoming Leader. State: Replica: 98f67b78652e49499a20f0cda78dfbf4, State: Running, Role: LEADER
I20260805 23:20:07.473138 15723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.474131 15712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: Neither blocks nor log segments found. Creating new log.
I20260805 23:20:07.474994 15712 log.cc:824] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: Log is configured to *not* fsync() on all Append() calls
I20260805 23:20:07.474992 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f67b78652e49499a20f0cda78dfbf4. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.475127 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:07.475728 15756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260805 23:20:07.476831 15756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260805 23:20:07.481021 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: No bootstrap required, opened a new log
I20260805 23:20:07.486778 15607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [term 1 FOLLOWER]: Refusing update from remote peer 98f67b78652e49499a20f0cda78dfbf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:20:07.487257 15721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.474359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.23.88.125:41935)
0805 23:20:07.475480 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.475486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.476430 (+ 944us) server_negotiation.cc:407] Connection header received
0805 23:20:07.476606 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.476610 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.476670 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.476759 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.478477 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.479278 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.480446 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.480780 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.480861 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.483061 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.483065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.483068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.483119 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.484944 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.485603 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.485608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.485611 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.485718 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.486061 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.486064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.486066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.486312 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.486417 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.486854 (+ 437us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.487015 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1013}
I20260805 23:20:07.487572 15749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:20:07.491891 15735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f67b78652e49499a20f0cda78dfbf4. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.492057 15735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.492836 15712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.493163 15712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e83cc2fe4c54ef5a4bcb753af000ae9, State: Initialized, Role: FOLLOWER
I20260805 23:20:07.493590 15712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } }
I20260805 23:20:07.495286 15712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: configured and running, proceeding with master startup.
I20260805 23:20:07.495332 15684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "98f67b78652e49499a20f0cda78dfbf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9"
I20260805 23:20:07.495632 15684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [term 0 FOLLOWER]: Advancing to term 1
I20260805 23:20:07.497437 15684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f67b78652e49499a20f0cda78dfbf4 in term 1.
I20260805 23:20:07.498438 15719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.475999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57050 (local address 127.23.88.126:34669)
0805 23:20:07.476106 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.476112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.487736 (+ 11624us) server_negotiation.cc:407] Connection header received
0805 23:20:07.488007 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.488010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.488066 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.488121 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.489297 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.490166 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.491269 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.491631 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.491720 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.494013 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.494017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.494019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.494053 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.496078 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.496791 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.496795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.496797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.496914 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.497254 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.497258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.497260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.497521 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.497642 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.498141 (+ 499us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.498280 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260805 23:20:07.498129 15683 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [term 1 FOLLOWER]: Refusing update from remote peer 98f67b78652e49499a20f0cda78dfbf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260805 23:20:07.500517 15749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.500662 15749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:07.501966 15723 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260805 23:20:07.502686 15756 catalog_manager.cc:1347] Generated new cluster ID: 0543524ad6ed45d5990d92736d566419
I20260805 23:20:07.502774 15756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260805 23:20:07.503966 15735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.504110 15735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.504470 15760 mvcc.cc:205] Tried to move back new op lower bound from 7315341342650032128 to 7315341342613483520. Current Snapshot: MvccSnapshot[applied={T|T < 7315341342650032128 or (T in {7315341342650032128})}]
I20260805 23:20:07.504704 15761 mvcc.cc:205] Tried to move back new op lower bound from 7315341342650032128 to 7315341342613483520. Current Snapshot: MvccSnapshot[applied={T|T < 7315341342650032128}]
I20260805 23:20:07.507543 15718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:07.475134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49598 (local address 127.23.88.124:33309)
0805 23:20:07.475292 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.475300 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.483425 (+ 8125us) server_negotiation.cc:407] Connection header received
0805 23:20:07.491253 (+ 7828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.491259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.491340 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.491438 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.497565 (+ 6127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.498399 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.500732 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:07.501122 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:07.501224 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:07.503503 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0805 23:20:07.503509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0805 23:20:07.503512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0805 23:20:07.503547 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0805 23:20:07.505471 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.506027 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.506032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.506034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.506128 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0805 23:20:07.506477 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0805 23:20:07.506482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0805 23:20:07.506487 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0805 23:20:07.506714 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0805 23:20:07.506861 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:07.507260 (+ 399us) server_negotiation.cc:299] Negotiation successful
0805 23:20:07.507394 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260805 23:20:07.508430 15775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260805 23:20:07.508519 15775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260805 23:20:07.519439 15763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.519634 15763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.519759 15763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f67b78652e49499a20f0cda78dfbf4. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.519802 15780 mvcc.cc:205] Tried to move back new op lower bound from 7315341342650032128 to 7315341342613483520. Current Snapshot: MvccSnapshot[applied={T|T < 7315341342650032128}]
I20260805 23:20:07.519855 15763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.519979 15763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.520069 15763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9 [sys.catalog]: This master's current role is: FOLLOWER
I20260805 23:20:07.530911 15723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98f67b78652e49499a20f0cda78dfbf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f67b78652e49499a20f0cda78dfbf4" member_type: VOTER last_known_addr { host: "127.23.88.126" port: 34669 } } peers { permanent_uuid: "3f7a7d8e8c8544e8bcbe04c906d36b99" member_type: VOTER last_known_addr { host: "127.23.88.125" port: 41935 } } peers { permanent_uuid: "5e83cc2fe4c54ef5a4bcb753af000ae9" member_type: VOTER last_known_addr { host: "127.23.88.124" port: 33309 } } }
I20260805 23:20:07.531119 15723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4 [sys.catalog]: This master's current role is: LEADER
I20260805 23:20:07.538823 15756 catalog_manager.cc:1370] Generated new certificate authority record
I20260805 23:20:07.540796 15756 catalog_manager.cc:1504] Loading token signing keys...
I20260805 23:20:07.565661 15756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98f67b78652e49499a20f0cda78dfbf4: Generated new TSK 0
I20260805 23:20:07.566635 15756 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260805 23:20:07.600720 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.597235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.23.88.125:41935)
0805 23:20:07.597375 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.597380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.599516 (+ 2136us) server_negotiation.cc:407] Connection header received
0805 23:20:07.599732 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.599736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.599791 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.599849 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.600570 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42774: BlockingRecv error: recv got EOF from 127.0.0.1:42774 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260805 23:20:07.602198 15719 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.598240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57058 (local address 127.23.88.126:34669)
0805 23:20:07.598363 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.598367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.601164 (+ 2797us) server_negotiation.cc:407] Connection header received
0805 23:20:07.601347 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.601350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.601404 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.601456 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.602067 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57058: BlockingRecv error: recv got EOF from 127.0.0.1:57058 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260805 23:20:07.603801 15718 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.598894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.23.88.124:33309)
0805 23:20:07.599027 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.599032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.602846 (+ 3814us) server_negotiation.cc:407] Connection header received
0805 23:20:07.603021 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.603025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.603100 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.603153 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.603671 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49604: BlockingRecv error: recv got EOF from 127.0.0.1:49604 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260805 23:20:07.609006 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.607253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.23.88.125:41935)
0805 23:20:07.607371 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.607376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.607949 (+ 573us) server_negotiation.cc:407] Connection header received
0805 23:20:07.608196 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.608199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.608284 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.608342 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.608891 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42788: BlockingRecv error: recv got EOF from 127.0.0.1:42788 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260805 23:20:07.614606 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.612919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.23.88.125:41935)
0805 23:20:07.613030 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.613034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.613606 (+ 572us) server_negotiation.cc:407] Connection header received
0805 23:20:07.613802 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.613805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.613859 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.613911 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.614481 (+ 570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42794: BlockingRecv error: recv got EOF from 127.0.0.1:42794 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260805 23:20:07.621701 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.619841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.23.88.125:41935)
0805 23:20:07.619971 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.619975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.620597 (+ 622us) server_negotiation.cc:407] Connection header received
0805 23:20:07.620827 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.620831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.620891 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.620947 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.621566 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42804: BlockingRecv error: recv got EOF from 127.0.0.1:42804 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260805 23:20:07.630187 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.628333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.23.88.125:41935)
0805 23:20:07.628509 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.628513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.629158 (+ 645us) server_negotiation.cc:407] Connection header received
0805 23:20:07.629371 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.629375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.629432 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.629484 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.630058 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42810: BlockingRecv error: recv got EOF from 127.0.0.1:42810 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
W20260805 23:20:07.643030 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.641175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.23.88.125:41935)
0805 23:20:07.641308 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.641313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.641969 (+ 656us) server_negotiation.cc:407] Connection header received
0805 23:20:07.642196 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.642200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.642258 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.642316 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.642910 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42824: BlockingRecv error: recv got EOF from 127.0.0.1:42824 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260805 23:20:07.664114 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.662005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42840 (local address 127.23.88.125:41935)
0805 23:20:07.662151 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.662157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.662891 (+ 734us) server_negotiation.cc:407] Connection header received
0805 23:20:07.663191 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.663195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.663260 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.663323 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.663970 (+ 647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42840: BlockingRecv error: recv got EOF from 127.0.0.1:42840 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260805 23:20:07.700855 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.699118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.23.88.125:41935)
0805 23:20:07.699261 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.699266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.699933 (+ 667us) server_negotiation.cc:407] Connection header received
0805 23:20:07.700106 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.700110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.700160 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.700253 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.700730 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42848: BlockingRecv error: recv got EOF from 127.0.0.1:42848 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260805 23:20:07.768888 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.767287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42852 (local address 127.23.88.125:41935)
0805 23:20:07.767486 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.767490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.767738 (+ 248us) server_negotiation.cc:407] Connection header received
0805 23:20:07.767953 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.767956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.768019 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.768096 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.768727 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42852: BlockingRecv error: recv got EOF from 127.0.0.1:42852 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260805 23:20:07.900641 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:07.899278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.23.88.125:41935)
0805 23:20:07.899472 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:07.899476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:07.899690 (+ 214us) server_negotiation.cc:407] Connection header received
0805 23:20:07.899792 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:07.899795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:07.899851 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:07.899990 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:07.900508 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42868: BlockingRecv error: recv got EOF from 127.0.0.1:42868 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260805 23:20:08.160283 15721 negotiation.cc:337] Failed RPC negotiation. Trace:
0805 23:20:08.159151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42874 (local address 127.23.88.125:41935)
0805 23:20:08.159281 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.159287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.159360 (+ 73us) server_negotiation.cc:407] Connection header received
0805 23:20:08.159548 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.159551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.159600 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.159692 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0805 23:20:08.160174 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42874: BlockingRecv error: recv got EOF from 127.0.0.1:42874 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260805 23:20:08.465488 15747 catalog_manager.cc:1259] Loaded cluster ID: 0543524ad6ed45d5990d92736d566419
I20260805 23:20:08.465580 15747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: loading cluster ID for follower catalog manager: success
I20260805 23:20:08.467448 15747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: acquiring CA information for follower catalog manager: success
I20260805 23:20:08.468701 15747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3f7a7d8e8c8544e8bcbe04c906d36b99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:20:08.509932 15775 catalog_manager.cc:1259] Loaded cluster ID: 0543524ad6ed45d5990d92736d566419
I20260805 23:20:08.510025 15775 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: loading cluster ID for follower catalog manager: success
I20260805 23:20:08.511950 15775 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: acquiring CA information for follower catalog manager: success
I20260805 23:20:08.513160 15775 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e83cc2fe4c54ef5a4bcb753af000ae9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260805 23:20:08.678471 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:08.674488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42880 (local address 127.23.88.125:41935)
0805 23:20:08.674748 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.674751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.674768 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:20:08.674878 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.674880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.675022 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.675186 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:08.675489 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.676022 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.676923 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.677243 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.677326 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:08.677651 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:08.678104 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:08.678196 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:08.678286 (+ 90us) server_negotiation.cc:299] Negotiation successful
0805 23:20:08.678340 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":110,"threads_started":1}
I20260805 23:20:08.685730 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:08.681783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.23.88.124:33309)
0805 23:20:08.682005 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.682008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.682059 (+ 51us) server_negotiation.cc:407] Connection header received
0805 23:20:08.682220 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.682222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.682343 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.682467 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:08.682638 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.683229 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.684157 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.684404 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.684491 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:08.684817 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:08.685274 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:08.685411 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:08.685508 (+ 97us) server_negotiation.cc:299] Negotiation successful
0805 23:20:08.685568 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260805 23:20:08.694097 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:08.689360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.23.88.125:41935)
0805 23:20:08.689514 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.689520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.689657 (+ 137us) server_negotiation.cc:407] Connection header received
0805 23:20:08.689900 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.689904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.689973 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.690067 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:08.690473 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.691280 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.692755 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.693120 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.693228 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:08.693537 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:08.693630 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:08.693704 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:08.693855 (+ 151us) server_negotiation.cc:299] Negotiation successful
0805 23:20:08.693921 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260805 23:20:08.694598 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:08.689321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57060 (local address 127.23.88.126:34669)
0805 23:20:08.689658 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.689663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.689680 (+ 17us) server_negotiation.cc:407] Connection header received
0805 23:20:08.689921 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.689924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.690123 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.690271 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:08.690497 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.691441 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.692802 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.693061 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.693182 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:08.693519 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:08.693911 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:08.694004 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:08.694368 (+ 364us) server_negotiation.cc:299] Negotiation successful
0805 23:20:08.694436 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":88,"threads_started":1}
I20260805 23:20:08.694612 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0805 23:20:08.689929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.23.88.124:33309)
0805 23:20:08.690071 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0805 23:20:08.690075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0805 23:20:08.690453 (+ 378us) server_negotiation.cc:407] Connection header received
0805 23:20:08.690650 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0805 23:20:08.690653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0805 23:20:08.690705 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0805 23:20:08.690783 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0805 23:20:08.691036 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.691954 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.693242 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0805 23:20:08.693613 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0805 23:20:08.693671 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0805 23:20:08.694070 (+ 399us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0805 23:20:08.694161 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0805 23:20:08.694209 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0805 23:20:08.694390 (+ 181us) server_negotiation.cc:299] Negotiation successful
0805 23:20:08.694450 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260805 23:20:08.698129 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15485
I20260805 23:20:08.705919 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15563
I20260805 23:20:08.714789 23905 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRmBmWz/build/debug/bin/kudu with pid 15641
2026-08-05T23:20:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3227 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6481 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292973 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST